I have put together a notebook on Multimodal RAG, where we do not process the documents with hefty pipelines but natively use:
text
I have put together a notebook on Multimodal RAG, where we do not process the documents with hefty pipelines but natively use:
new_line
- 
text
- 
https://huggingface.co/vidore/colpali
vidore/colpali
model
resource
https://huggingface.co/vidore/colpali
 for retrieval 📖 it doesn't need indexing with image-text pairs but just images!
text
 for retrieval 📖 it doesn't need indexing with image-text pairs but just images!
new_line
- 
text
- 
https://huggingface.co/Qwen/Qwen2-VL-2B-Instruct
Qwen/Qwen2-VL-2B-Instruct
model
resource
https://huggingface.co/Qwen/Qwen2-VL-2B-Instruct
 for generation 💬 directly feed images as is to a vision language model with no processing to text! 
text
 for generation 💬 directly feed images as is to a vision language model with no processing to text! 
new_line
I used ColPali implementation of the new 🐭 Byaldi library by 
text
I used ColPali implementation of the new 🐭 Byaldi library by 
@bclavie
mention
bclavie
  🤗
text
  🤗
new_line
https://github.com/answerdotai/byaldi
https://github.com/answerdotai/byaldi
link
new_line
Link to notebook: 
text
Link to notebook: 
https://github.com/merveenoyan/smol-vision/blob/main/ColPali_%2B_Qwen2_VL.ipynb
https://github.com/merveenoyan/smol-vision/blob/main/ColPali_%2B_Qwen2_VL.ipynb
link
new_line
I have put together a notebook on Multimodal RAG, where we do not process the documents with hefty pipelines but natively use:
- https://huggingface.co/vidore/colpali for retrieval 📖 it doesn't need indexing with image-text pairs but just images!
- https://huggingface.co/Qwen/Qwen2-VL-2B-Instruct for generation 💬 directly feed images as is to a vision language model with no processing to text! 
I used ColPali implementation of the new 🐭 Byaldi library by @bclavie  🤗
https://github.com/answerdotai/byaldi
Link to notebook: https://github.com/merveenoyan/smol-vision/blob/main/ColPali_%2B_Qwen2_VL.ipynb

🔥
rwightman umair894 clem louisbrulenaudet s3nh John6666 BasitMustafa Johnyquest7 phanhoang resbyte denizaybey allandclive fdaudens vilarin ak0601 rreed-pha xi0v Rajaram1996 Rayvee oceansweep parjun byteprobe Filippo
👍
hitchhiker3010 Csplk sasikiran fsommers rogermt navin7 sambarnett96 oceansweep ysdede shreyamondal
❤️
rreed-pha oceansweep Yassmen madstuntman11