📢Delighted to share personal findings  🔎 East Asian LLM Qwen1.5 🇨🇳 reasoning capabilities 🧠 Target Sentiment Analysis (TSA).  Starting with the one of the smallest Qwen1.5-1.8B-Chat version, for the original Eastern-European texts (🇷🇺) and their translated versions (🇺🇸) in zero-shot-learning mode setup, the key takeaways of such experiments were as follows:
text
📢Delighted to share personal findings  🔎 East Asian LLM Qwen1.5 🇨🇳 reasoning capabilities 🧠 Target Sentiment Analysis (TSA).  Starting with the one of the smallest Qwen1.5-1.8B-Chat version, for the original Eastern-European texts (🇷🇺) and their translated versions (🇺🇸) in zero-shot-learning mode setup, the key takeaways of such experiments were as follows:
new_line
new_line
✅ 1. Model is capable to perform reasoning in Eastern Eropean languages (🇷🇺) (remember it is 1.8B), switching to Qwen2 results in strong improvement, with results that surpasses LLaMA2-70B-chat (more on difference below). 
text
✅ 1. Model is capable to perform reasoning in Eastern Eropean languages (🇷🇺) (remember it is 1.8B), switching to Qwen2 results in strong improvement, with results that surpasses LLaMA2-70B-chat (more on difference below). 
new_line
✅ 2. Despite the size of 1.8B, reasoning in English has a significant gap in underpeforming (F1=~34%) to the most closest Flan-T5-XL (2.8B) which showcases F1=43%.
text
✅ 2. Despite the size of 1.8B, reasoning in English has a significant gap in underpeforming (F1=~34%) to the most closest Flan-T5-XL (2.8B) which showcases F1=43%.
new_line
new_line
💡 The most intriguing fact that Qwen1.5-1.8B-Chat: 
text
💡 The most intriguing fact that Qwen1.5-1.8B-Chat: 
new_line
it generates new words in Russian I've never seen before: "неретеневая" (negativitively), and imputes the entries in Chinese. The reason of such a low results, is that model was not been able to follow the input instruction and shares all the opinions per each class.  All of that has been improved though in Qwen2-1.5B.
text
it generates new words in Russian I've never seen before: "неретеневая" (negativitively), and imputes the entries in Chinese. The reason of such a low results, is that model was not been able to follow the input instruction and shares all the opinions per each class.  All of that has been improved though in Qwen2-1.5B.
new_line
new_line
Benchmark: 
text
Benchmark: 
https://github.com/nicolay-r/RuSentNE-LLM-Benchmark
https://github.com/nicolay-r/RuSentNE-LLM-Benchmark
link
new_line
Model: 
text
Model: 
https://huggingface.co/Qwen/Qwen1.5-1.8B-Chat
Qwen/Qwen1.5-1.8B-Chat
model
resource
https://huggingface.co/Qwen/Qwen1.5-1.8B-Chat
new_line
Dataset: 
text
Dataset: 
https://github.com/dialogue-evaluation/RuSentNE-evaluation
https://github.com/dialogue-evaluation/RuSentNE-evaluation
link
new_line
Related paper: Large Language Models in Targeted Sentiment Analysis (2404.12342)
text
Related paper: Large Language Models in Targeted Sentiment Analysis (2404.12342)
new_line
Collection: 
text
Collection: 
https://huggingface.co/collections/nicolay-r/sentiment-analysis-665ba391e0eba729021ea101
https://huggingface.co/collections/nicolay-r/sentiment-analysis-665ba391e0eba729021ea101
link
📢Delighted to share personal findings  🔎 East Asian LLM Qwen1.5 🇨🇳 reasoning capabilities 🧠 Target Sentiment Analysis (TSA).  Starting with the one of the smallest Qwen1.5-1.8B-Chat version, for the original Eastern-European texts (🇷🇺) and their translated versions (🇺🇸) in zero-shot-learning mode setup, the key takeaways of such experiments were as follows:

✅ 1. Model is capable to perform reasoning in Eastern Eropean languages (🇷🇺) (remember it is 1.8B), switching to Qwen2 results in strong improvement, with results that surpasses LLaMA2-70B-chat (more on difference below). 
✅ 2. Despite the size of 1.8B, reasoning in English has a significant gap in underpeforming (F1=~34%) to the most closest Flan-T5-XL (2.8B) which showcases F1=43%.

💡 The most intriguing fact that Qwen1.5-1.8B-Chat: 
it generates new words in Russian I've never seen before: "неретеневая" (negativitively), and imputes the entries in Chinese. The reason of such a low results, is that model was not been able to follow the input instruction and shares all the opinions per each class.  All of that has been improved though in Qwen2-1.5B.

Benchmark: https://github.com/nicolay-r/RuSentNE-LLM-Benchmark
Model: https://huggingface.co/Qwen/Qwen1.5-1.8B-Chat
Dataset: https://github.com/dialogue-evaluation/RuSentNE-evaluation
Related paper: Large Language Models in Targeted Sentiment Analysis (2404.12342)
Collection: https://huggingface.co/collections/nicolay-r/sentiment-analysis-665ba391e0eba729021ea101
👍
victor kristaller486 louisbrulenaudet ZeroWw