Hi! I'm having an issue converting a Phi3 model to a torch scripted model.  Code, error, and versions below.  Any guidance would be super helpful, thanks!
text
Hi! I'm having an issue converting a Phi3 model to a torch scripted model.  Code, error, and versions below.  Any guidance would be super helpful, thanks!
new_line
new_line
Here is the code:
text
Here is the code:
new_line
new_line
from transformers import AutoModelForCausalLM
import torch

base_model = "microsoft/Phi-3-mini-4k-instruct"

model = AutoModelForCausalLM.from_pretrained(
        base_model,
        return_dict=True,
        low_cpu_mem_usage=True,
        torch_dtype=torch.float16,
        device_map="auto",
        trust_remote_code=True,
        torchscript=True
)

scripted_model = torch.jit.script(model)
```
from transformers import AutoModelForCausalLM
import torch

base_model = "microsoft/Phi-3-mini-4k-instruct"

model = AutoModelForCausalLM.from_pretrained(
        base_model,
        return_dict=True,
        low_cpu_mem_usage=True,
        torch_dtype=torch.float16,
        device_map="auto",
        trust_remote_code=True,
        torchscript=True
)

scripted_model = torch.jit.script(model)
```
code_fence
new_line
new_line
Here is the error:
text
Here is the error:
new_line
new_line
 File ".../anaconda3/envs/qcomm_phi3.5instruct_aihub_py_310/lib/python3.10/site-packages/torch/jit/frontend.py", line 910, in <listcomp>
    args = [build_expr(ctx, py_arg) for py_arg in expr.args]
  File ".../anaconda3/envs/qcomm_phi3.5instruct_aihub_py_310/lib/python3.10/site-packages/torch/jit/frontend.py", line 417, in __call__
    return method(ctx, node)
  File ".../anaconda3/envs/qcomm_phi3.5instruct_aihub_py_310/lib/python3.10/site-packages/torch/jit/frontend.py", line 1255, in build_GeneratorExp
    return ExprBuilder.build_ListComp(ctx, stmt)
  File ".../anaconda3/envs/qcomm_phi3.5instruct_aihub_py_310/lib/python3.10/site-packages/torch/jit/frontend.py", line 1244, in build_ListComp
    raise NotSupportedError(r, "Comprehension ifs are not supported yet")
torch.jit.frontend.NotSupportedError: Comprehension ifs are not supported yet:
  File ".../.cache/huggingface/modules/transformers_modules/microsoft/Phi-3-mini-4k-instruct/0a67737cc96d2554230f90338b163bc6380a2a85/modeling_phi3.py", line 1148
            next_cache = next_decoder_cache.to_legacy_cache() if use_legacy_cache else next_decoder_cache
        if not return_dict:
            return tuple(v for v in [hidden_states, next_cache, all_hidden_states, all_self_attns] if v is not None)
        return BaseModelOutputWithPast(
            last_hidden_state=hidden_states
```
 File ".../anaconda3/envs/qcomm_phi3.5instruct_aihub_py_310/lib/python3.10/site-packages/torch/jit/frontend.py", line 910, in <listcomp>
    args = [build_expr(ctx, py_arg) for py_arg in expr.args]
  File ".../anaconda3/envs/qcomm_phi3.5instruct_aihub_py_310/lib/python3.10/site-packages/torch/jit/frontend.py", line 417, in __call__
    return method(ctx, node)
  File ".../anaconda3/envs/qcomm_phi3.5instruct_aihub_py_310/lib/python3.10/site-packages/torch/jit/frontend.py", line 1255, in build_GeneratorExp
    return ExprBuilder.build_ListComp(ctx, stmt)
  File ".../anaconda3/envs/qcomm_phi3.5instruct_aihub_py_310/lib/python3.10/site-packages/torch/jit/frontend.py", line 1244, in build_ListComp
    raise NotSupportedError(r, "Comprehension ifs are not supported yet")
torch.jit.frontend.NotSupportedError: Comprehension ifs are not supported yet:
  File ".../.cache/huggingface/modules/transformers_modules/microsoft/Phi-3-mini-4k-instruct/0a67737cc96d2554230f90338b163bc6380a2a85/modeling_phi3.py", line 1148
            next_cache = next_decoder_cache.to_legacy_cache() if use_legacy_cache else next_decoder_cache
        if not return_dict:
            return tuple(v for v in [hidden_states, next_cache, all_hidden_states, all_self_attns] if v is not None)
        return BaseModelOutputWithPast(
            last_hidden_state=hidden_states
```
code_fence
new_line
new_line
torch==2.5.1
text
torch==2.5.1
new_line
transformers==4.51.3
text
transformers==4.51.3
Hi! I'm having an issue converting a Phi3 model to a torch scripted model.  Code, error, and versions below.  Any guidance would be super helpful, thanks!

Here is the code:

```
from transformers import AutoModelForCausalLM
import torch

base_model = "microsoft/Phi-3-mini-4k-instruct"

model = AutoModelForCausalLM.from_pretrained(
        base_model,
        return_dict=True,
        low_cpu_mem_usage=True,
        torch_dtype=torch.float16,
        device_map="auto",
        trust_remote_code=True,
        torchscript=True
)

scripted_model = torch.jit.script(model)
```

Here is the error:

```
 File ".../anaconda3/envs/qcomm_phi3.5instruct_aihub_py_310/lib/python3.10/site-packages/torch/jit/frontend.py", line 910, in <listcomp>
    args = [build_expr(ctx, py_arg) for py_arg in expr.args]
  File ".../anaconda3/envs/qcomm_phi3.5instruct_aihub_py_310/lib/python3.10/site-packages/torch/jit/frontend.py", line 417, in __call__
    return method(ctx, node)
  File ".../anaconda3/envs/qcomm_phi3.5instruct_aihub_py_310/lib/python3.10/site-packages/torch/jit/frontend.py", line 1255, in build_GeneratorExp
    return ExprBuilder.build_ListComp(ctx, stmt)
  File ".../anaconda3/envs/qcomm_phi3.5instruct_aihub_py_310/lib/python3.10/site-packages/torch/jit/frontend.py", line 1244, in build_ListComp
    raise NotSupportedError(r, "Comprehension ifs are not supported yet")
torch.jit.frontend.NotSupportedError: Comprehension ifs are not supported yet:
  File ".../.cache/huggingface/modules/transformers_modules/microsoft/Phi-3-mini-4k-instruct/0a67737cc96d2554230f90338b163bc6380a2a85/modeling_phi3.py", line 1148
            next_cache = next_decoder_cache.to_legacy_cache() if use_legacy_cache else next_decoder_cache
        if not return_dict:
            return tuple(v for v in [hidden_states, next_cache, all_hidden_states, all_self_attns] if v is not None)
        return BaseModelOutputWithPast(
            last_hidden_state=hidden_states
```

torch==2.5.1
transformers==4.51.3
👀
John6666
6640bbd0220cfa8cbfdce080
https://cdn-avatars.huggingface.co/v1/production/uploads/6640bbd0220cfa8cbfdce080/wiAHUu5ewawyipNs0YFBR.png
John Smith
John6666
user
66be647e70f0b1935a709623
/avatars/c99bf0f3a4518cad3b1c9bc6f6053328.svg
George Williams
sourcesync
user