Spaces:
Sleeping
Sleeping
Download run_embedding.py from Subhadip007/researchpilot-api: direct link, hf CLI and curl.
- Browser
- Download file 901 Bytes
-
https://huggingface.co/spaces/Subhadip007/researchpilot-api/resolve/32cff2fe35cc30459a152fcc741e0401a4dff1a6/run_embedding.py
- Command line
-
hf download hf://spaces/Subhadip007/researchpilot-api@32cff2fe35cc30459a152fcc741e0401a4dff1a6/run_embedding.py
-
curl -L -o run_embedding.py https://huggingface.co/spaces/Subhadip007/researchpilot-api/resolve/32cff2fe35cc30459a152fcc741e0401a4dff1a6/run_embedding.py
901 Bytes
| """ | |
| Phase 6: Generate embeddings for all semantic chunks. | |
| Run from project root: | |
| python run_embedding.py | |
| Input: data/chunks/*_semantic.json (15,664 chunks) | |
| Output: data/embeddings/embeddings.npy (shape: 15664 x 768) | |
| data/embeddings/chunk_ids.npy | |
| data/embeddings/embedding_index.json | |
| data/embeddings/chunk_metadata.json | |
| """ | |
| from src.utils.logger import setup_logger, get_logger | |
| from src.embeddings.embedding_pipeline import EmbeddingPipeline | |
| setup_logger() | |
| logger = get_logger(__name__) | |
| def main(): | |
| logger.info("=" * 60) | |
| logger.info("PHASE 6 — EMBEDDING PIPELINE") | |
| logger.info("=" * 60) | |
| pipeline = EmbeddingPipeline() | |
| stats = pipeline.run() | |
| logger.info("=" * 60) | |
| logger.info("EMBEDDING COMPLETE") | |
| for k, v in stats.items(): | |
| logger.info(f" {k}: {v}") | |
| logger.info("=" * 60) | |
| if __name__ == "__main__": | |
| main() |