Spaces:

---

GhadaSaylami

/

kalim-rag-test

like 0

Running on Zero

Thanks for merging #1 — I had not seen it land when I opened this, so please
ignore the line at the top of the original description saying this replaces it.
It does not. This adds one more fix on top of what you already merged.

I tried to add it to #1 directly, but Hugging Face only lets the repo owner push
into an existing PR ref, so it had to be a new PR.

The one change here: the ZeroGPU CUDA error

The index build moved off the import path in #1, so the Space serves the UI
immediately instead of waiting for a 471MB model download on every restart.
That exposed this on the background thread:

Low-level CUDA init (torch._C._cuda_init) reached. This means ZeroGPU's
PyTorch CUDA emulation mode did not intercept a CUDA operation in your code.

sentence-transformers picks a device by asking torch whether CUDA is
available. On ZeroGPU that answers yes — CUDA is emulated, and only intercepted
for calls made on the request path inside an @spaces.GPU function. A build on
a background thread is outside that path, so the call reaches
torch._C._cuda_init directly and the build dies.

The embedding model is now pinned to the CPU explicitly, which skips the
availability check rather than fighting the emulation. Nothing here wants a GPU:
it is a MiniLM embedding 95 short documents once per restart, which the CPU
finishes in seconds. KALIM_EMBEDDING_DEVICE overrides it for a machine with a
GPU worth using.

Both Chroma paths are covered — build_index and search each embed through
this model and hand Chroma the finished vectors, so Chroma never runs an
embedding function of its own.

sentence-transformers is not a test dependency, so the new tests stub the
constructor; what they pin down is that a device is passed at all, and that it
is the CPU by default. 151 tests pass.

Since #1 is merged, the only files that actually differ here are
config/settings.py, src/rag/embeddings.py and the new test.