Create immediate_feed.txt

immediate_feed.txt

@@ -0,0 +1,383 @@
+============================================================
+CantorLinear ImageNet CLIP Features Training
+============================================================
+
+Configuration:
+  Dataset: AbstractPhil/imagenet-clip-features-orderly
+  CLIP dim: 512
+  Hidden dims: Direct
+  Cantor depth: 8
+  Batch size: 512
+  Learning rate: 0.001
+  Device: cuda
+
+Loading dataset...
+clip_vit_b16/train-00000-of-00006.parque(…): 100%
+ 571M/571M [00:09<00:00, 192MB/s]
+clip_vit_b16/train-00001-of-00006.parque(…): 100%
+ 571M/571M [00:05<00:00, 144MB/s]
+clip_vit_b16/train-00002-of-00006.parque(…): 100%
+ 571M/571M [00:09<00:00, 59.9MB/s]
+clip_vit_b16/train-00003-of-00006.parque(…): 100%
+ 571M/571M [00:08<00:00, 116MB/s]
+clip_vit_b16/train-00004-of-00006.parque(…): 100%
+ 571M/571M [00:07<00:00, 194MB/s]
+clip_vit_b16/train-00005-of-00006.parque(…): 100%
+ 571M/571M [00:08<00:00, 127MB/s]
+clip_vit_b16/validation-00000-of-00001.p(…): 100%
+ 134M/134M [00:03<00:00, 98.9MB/s]
+clip_vit_b16/test-00000-of-00001.parquet: 100%
+ 267M/267M [00:05<00:00, 158MB/s]
+Generating train split: 100%
+ 1281167/1281167 [00:08<00:00, 150853.22 examples/s]
+Generating validation split: 100%
+ 50000/50000 [00:02<00:00, 18659.96 examples/s]
+Generating test split: 100%
+ 100000/100000 [00:00<00:00, 178346.18 examples/s]
+Train samples: 1153050
+Val samples: 128117
+
+Building model...
+Total parameters: 513,001
+Trainable parameters: 513,001
+CantorLinear layers: 1
+Avg mask density: 0.0391
+
+Starting training...
+Epoch 1/50: 100%|██████████| 2253/2253 [01:59<00:00, 18.89it/s, loss=6.8454, acc=14.31%]
+Evaluating: 100%|██████████| 251/251 [00:13<00:00, 18.88it/s]
+
+Epoch 1/50
+  Train Loss: 6.8453 | Train Acc: 14.32%
+  Val Loss: 6.7203 | Val Acc: 34.11%
+  Mean Alpha: 0.5588 | Mean Density: 0.0391
+  ✓ New best model saved! (Val Acc: 34.11%)
+Epoch 2/50: 100%|██████████| 2253/2253 [01:58<00:00, 19.03it/s, loss=6.4762, acc=37.85%]
+Evaluating: 100%|██████████| 251/251 [00:13<00:00, 18.86it/s]
+
+Epoch 2/50
+  Train Loss: 6.4759 | Train Acc: 37.85%
+  Val Loss: 6.1765 | Val Acc: 39.15%
+  Mean Alpha: 0.5812 | Mean Density: 0.0391
+  ✓ New best model saved! (Val Acc: 39.15%)
+Epoch 3/50: 100%|██████████| 2253/2253 [01:58<00:00, 19.07it/s, loss=5.7757, acc=40.62%]
+Evaluating: 100%|██████████| 251/251 [00:13<00:00, 18.74it/s]
+
+Epoch 3/50
+  Train Loss: 5.7754 | Train Acc: 40.62%
+  Val Loss: 5.3353 | Val Acc: 42.73%
+  Mean Alpha: 0.6131 | Mean Density: 0.0391
+  ✓ New best model saved! (Val Acc: 42.73%)
+Epoch 4/50: 100%|██████████| 2253/2253 [01:57<00:00, 19.10it/s, loss=4.8301, acc=46.62%]
+Evaluating: 100%|██████████| 251/251 [00:13<00:00, 18.50it/s]
+
+Epoch 4/50
+  Train Loss: 4.8296 | Train Acc: 46.62%
+  Val Loss: 4.3151 | Val Acc: 51.32%
+  Mean Alpha: 0.6513 | Mean Density: 0.0391
+  ✓ New best model saved! (Val Acc: 51.32%)
+Epoch 5/50: 100%|██████████| 2253/2253 [01:57<00:00, 19.11it/s, loss=3.7933, acc=56.48%]
+Evaluating: 100%|██████████| 251/251 [00:13<00:00, 19.13it/s]
+
+Epoch 5/50
+  Train Loss: 3.7930 | Train Acc: 56.48%
+  Val Loss: 3.2999 | Val Acc: 61.08%
+  Mean Alpha: 0.6914 | Mean Density: 0.0391
+  ✓ New best model saved! (Val Acc: 61.08%)
+Epoch 6/50: 100%|██████████| 2253/2253 [01:58<00:00, 19.03it/s, loss=2.8774, acc=64.57%]
+Evaluating: 100%|██████████| 251/251 [00:13<00:00, 18.69it/s]
+
+Epoch 6/50
+  Train Loss: 2.8771 | Train Acc: 64.57%
+  Val Loss: 2.5271 | Val Acc: 67.17%
+  Mean Alpha: 0.7286 | Mean Density: 0.0391
+  ✓ New best model saved! (Val Acc: 67.17%)
+Epoch 7/50: 100%|██████████| 2253/2253 [01:58<00:00, 19.03it/s, loss=2.2531, acc=68.98%]
+Evaluating: 100%|██████████| 251/251 [00:13<00:00, 18.88it/s]
+
+Epoch 7/50
+  Train Loss: 2.2527 | Train Acc: 68.98%
+  Val Loss: 2.0361 | Val Acc: 70.25%
+  Mean Alpha: 0.7614 | Mean Density: 0.0391
+  ✓ New best model saved! (Val Acc: 70.25%)
+Epoch 8/50: 100%|██████████| 2253/2253 [01:58<00:00, 19.08it/s, loss=1.8587, acc=71.36%]
+Evaluating: 100%|██████████| 251/251 [00:13<00:00, 18.40it/s]
+
+Epoch 8/50
+  Train Loss: 1.8588 | Train Acc: 71.36%
+  Val Loss: 1.7251 | Val Acc: 71.92%
+  Mean Alpha: 0.7910 | Mean Density: 0.0391
+  ✓ New best model saved! (Val Acc: 71.92%)
+Epoch 9/50: 100%|██████████| 2253/2253 [01:58<00:00, 19.05it/s, loss=1.6052, acc=72.79%]
+Evaluating: 100%|██████████| 251/251 [00:13<00:00, 18.70it/s]
+
+Epoch 9/50
+  Train Loss: 1.6050 | Train Acc: 72.79%
+  Val Loss: 1.5215 | Val Acc: 73.01%
+  Mean Alpha: 0.8171 | Mean Density: 0.0391
+  ✓ New best model saved! (Val Acc: 73.01%)
+Epoch 10/50: 100%|██████████| 2253/2253 [01:57<00:00, 19.17it/s, loss=1.4355, acc=73.78%]
+Evaluating: 100%|██████████| 251/251 [00:13<00:00, 18.74it/s]
+
+Epoch 10/50
+  Train Loss: 1.4355 | Train Acc: 73.78%
+  Val Loss: 1.3817 | Val Acc: 73.82%
+  Mean Alpha: 0.8400 | Mean Density: 0.0391
+  ✓ New best model saved! (Val Acc: 73.82%)
+Epoch 11/50: 100%|██████████| 2253/2253 [01:57<00:00, 19.19it/s, loss=1.3162, acc=74.56%]
+Evaluating: 100%|██████████| 251/251 [00:13<00:00, 18.31it/s]
+
+Epoch 11/50
+  Train Loss: 1.3162 | Train Acc: 74.56%
+  Val Loss: 1.2818 | Val Acc: 74.49%
+  Mean Alpha: 0.8598 | Mean Density: 0.0391
+  ✓ New best model saved! (Val Acc: 74.49%)
+Epoch 12/50: 100%|██████████| 2253/2253 [01:57<00:00, 19.12it/s, loss=1.2289, acc=75.16%]
+Evaluating: 100%|██████████| 251/251 [00:13<00:00, 18.80it/s]
+
+Epoch 12/50
+  Train Loss: 1.2287 | Train Acc: 75.16%
+  Val Loss: 1.2072 | Val Acc: 74.97%
+  Mean Alpha: 0.8766 | Mean Density: 0.0391
+  ✓ New best model saved! (Val Acc: 74.97%)
+Epoch 13/50: 100%|██████████| 2253/2253 [01:57<00:00, 19.13it/s, loss=1.1623, acc=75.65%]
+Evaluating: 100%|██████████| 251/251 [00:13<00:00, 18.78it/s]
+
+Epoch 13/50
+  Train Loss: 1.1622 | Train Acc: 75.65%
+  Val Loss: 1.1495 | Val Acc: 75.39%
+  Mean Alpha: 0.8909 | Mean Density: 0.0391
+  ✓ New best model saved! (Val Acc: 75.39%)
+Epoch 14/50: 100%|██████████| 2253/2253 [01:58<00:00, 19.08it/s, loss=1.1100, acc=76.09%]
+Evaluating: 100%|██████████| 251/251 [00:13<00:00, 19.00it/s]
+
+Epoch 14/50
+  Train Loss: 1.1100 | Train Acc: 76.09%
+  Val Loss: 1.1040 | Val Acc: 75.75%
+  Mean Alpha: 0.9027 | Mean Density: 0.0391
+  ✓ New best model saved! (Val Acc: 75.75%)
+Epoch 15/50: 100%|██████████| 2253/2253 [01:57<00:00, 19.15it/s, loss=1.0681, acc=76.46%]
+Evaluating: 100%|██████████| 251/251 [00:13<00:00, 18.85it/s]
+
+Epoch 15/50
+  Train Loss: 1.0681 | Train Acc: 76.46%
+  Val Loss: 1.0670 | Val Acc: 76.06%
+  Mean Alpha: 0.9128 | Mean Density: 0.0391
+  ✓ New best model saved! (Val Acc: 76.06%)
+Epoch 16/50: 100%|██████████| 2253/2253 [01:57<00:00, 19.11it/s, loss=1.0336, acc=76.79%]
+Evaluating: 100%|██████████| 251/251 [00:13<00:00, 18.90it/s]
+
+Epoch 16/50
+  Train Loss: 1.0337 | Train Acc: 76.79%
+  Val Loss: 1.0367 | Val Acc: 76.32%
+  Mean Alpha: 0.9212 | Mean Density: 0.0391
+  ✓ New best model saved! (Val Acc: 76.32%)
+Epoch 17/50: 100%|██████████| 2253/2253 [01:57<00:00, 19.13it/s, loss=1.0049, acc=77.06%]
+Evaluating: 100%|██████████| 251/251 [00:13<00:00, 18.68it/s]
+
+Epoch 17/50
+  Train Loss: 1.0048 | Train Acc: 77.07%
+  Val Loss: 1.0113 | Val Acc: 76.57%
+  Mean Alpha: 0.9284 | Mean Density: 0.0391
+  ✓ New best model saved! (Val Acc: 76.57%)
+Epoch 18/50: 100%|██████████| 2253/2253 [01:57<00:00, 19.22it/s, loss=0.9806, acc=77.32%]
+Evaluating: 100%|██████████| 251/251 [00:13<00:00, 18.67it/s]
+
+Epoch 18/50
+  Train Loss: 0.9809 | Train Acc: 77.32%
+  Val Loss: 0.9898 | Val Acc: 76.79%
+  Mean Alpha: 0.9343 | Mean Density: 0.0391
+  ✓ New best model saved! (Val Acc: 76.79%)
+Epoch 19/50: 100%|██████████| 2253/2253 [01:57<00:00, 19.09it/s, loss=0.9598, acc=77.55%]
+Evaluating: 100%|██████████| 251/251 [00:13<00:00, 18.82it/s]
+
+Epoch 19/50
+  Train Loss: 0.9598 | Train Acc: 77.55%
+  Val Loss: 0.9715 | Val Acc: 76.96%
+  Mean Alpha: 0.9396 | Mean Density: 0.0391
+  ✓ New best model saved! (Val Acc: 76.96%)
+Epoch 20/50: 100%|██████████| 2253/2253 [01:57<00:00, 19.15it/s, loss=0.9419, acc=77.75%]
+Evaluating: 100%|██████████| 251/251 [00:13<00:00, 18.83it/s]
+
+Epoch 20/50
+  Train Loss: 0.9419 | Train Acc: 77.75%
+  Val Loss: 0.9556 | Val Acc: 77.15%
+  Mean Alpha: 0.9440 | Mean Density: 0.0391
+  ✓ New best model saved! (Val Acc: 77.15%)
+Epoch 21/50: 100%|██████████| 2253/2253 [01:57<00:00, 19.21it/s, loss=0.9264, acc=77.92%]
+Evaluating: 100%|██████████| 251/251 [00:13<00:00, 18.74it/s]
+
+Epoch 21/50
+  Train Loss: 0.9265 | Train Acc: 77.92%
+  Val Loss: 0.9420 | Val Acc: 77.27%
+  Mean Alpha: 0.9477 | Mean Density: 0.0391
+  ✓ New best model saved! (Val Acc: 77.27%)
+Epoch 22/50: 100%|██████████| 2253/2253 [01:56<00:00, 19.26it/s, loss=0.9128, acc=78.09%]
+Evaluating: 100%|██████████| 251/251 [00:13<00:00, 18.94it/s]
+
+Epoch 22/50
+  Train Loss: 0.9126 | Train Acc: 78.09%
+  Val Loss: 0.9300 | Val Acc: 77.41%
+  Mean Alpha: 0.9511 | Mean Density: 0.0391
+  ✓ New best model saved! (Val Acc: 77.41%)
+Epoch 23/50: 100%|██████████| 2253/2253 [01:57<00:00, 19.23it/s, loss=0.9007, acc=78.24%]
+Evaluating: 100%|██████████| 251/251 [00:13<00:00, 18.52it/s]
+
+Epoch 23/50
+  Train Loss: 0.9007 | Train Acc: 78.24%
+  Val Loss: 0.9195 | Val Acc: 77.54%
+  Mean Alpha: 0.9540 | Mean Density: 0.0391
+  ✓ New best model saved! (Val Acc: 77.54%)
+Epoch 24/50: 100%|██████████| 2253/2253 [01:57<00:00, 19.17it/s, loss=0.8902, acc=78.37%]
+Evaluating: 100%|██████████| 251/251 [00:13<00:00, 18.95it/s]
+
+Epoch 24/50
+  Train Loss: 0.8904 | Train Acc: 78.37%
+  Val Loss: 0.9102 | Val Acc: 77.63%
+  Mean Alpha: 0.9565 | Mean Density: 0.0391
+  ✓ New best model saved! (Val Acc: 77.63%)
+Epoch 25/50: 100%|██████████| 2253/2253 [01:58<00:00, 19.03it/s, loss=0.8809, acc=78.49%]
+Evaluating: 100%|██████████| 251/251 [00:13<00:00, 19.09it/s]
+
+Epoch 25/50
+  Train Loss: 0.8809 | Train Acc: 78.49%
+  Val Loss: 0.9020 | Val Acc: 77.70%
+  Mean Alpha: 0.9587 | Mean Density: 0.0391
+  ✓ New best model saved! (Val Acc: 77.70%)
+Epoch 26/50: 100%|██████████| 2253/2253 [01:57<00:00, 19.18it/s, loss=0.8725, acc=78.60%]
+Evaluating: 100%|██████████| 251/251 [00:13<00:00, 18.63it/s]
+
+Epoch 26/50
+  Train Loss: 0.8724 | Train Acc: 78.60%
+  Val Loss: 0.8949 | Val Acc: 77.78%
+  Mean Alpha: 0.9606 | Mean Density: 0.0391
+  ✓ New best model saved! (Val Acc: 77.78%)
+Epoch 27/50: 100%|██████████| 2253/2253 [01:57<00:00, 19.13it/s, loss=0.8651, acc=78.71%]
+Evaluating: 100%|██████████| 251/251 [00:13<00:00, 18.61it/s]
+
+Epoch 27/50
+  Train Loss: 0.8651 | Train Acc: 78.71%
+  Val Loss: 0.8885 | Val Acc: 77.87%
+  Mean Alpha: 0.9623 | Mean Density: 0.0391
+  ✓ New best model saved! (Val Acc: 77.87%)
+Epoch 28/50: 100%|██████████| 2253/2253 [01:57<00:00, 19.20it/s, loss=0.8585, acc=78.79%]
+Evaluating: 100%|██████████| 251/251 [00:13<00:00, 18.69it/s]
+
+Epoch 28/50
+  Train Loss: 0.8584 | Train Acc: 78.79%
+  Val Loss: 0.8827 | Val Acc: 77.92%
+  Mean Alpha: 0.9637 | Mean Density: 0.0391
+  ✓ New best model saved! (Val Acc: 77.92%)
+Epoch 29/50: 100%|██████████| 2253/2253 [01:57<00:00, 19.20it/s, loss=0.8527, acc=78.87%]
+Evaluating: 100%|██████████| 251/251 [00:13<00:00, 18.76it/s]
+
+Epoch 29/50
+  Train Loss: 0.8526 | Train Acc: 78.87%
+  Val Loss: 0.8778 | Val Acc: 77.97%
+  Mean Alpha: 0.9650 | Mean Density: 0.0391
+  ✓ New best model saved! (Val Acc: 77.97%)
+Epoch 30/50: 100%|██████████| 2253/2253 [01:57<00:00, 19.13it/s, loss=0.8475, acc=78.95%]
+Evaluating: 100%|██████████| 251/251 [00:13<00:00, 18.94it/s]
+
+Epoch 30/50
+  Train Loss: 0.8476 | Train Acc: 78.95%
+  Val Loss: 0.8733 | Val Acc: 78.03%
+  Mean Alpha: 0.9661 | Mean Density: 0.0391
+  ✓ New best model saved! (Val Acc: 78.03%)
+Epoch 31/50: 100%|██████████| 2253/2253 [01:57<00:00, 19.13it/s, loss=0.8429, acc=79.02%]
+Evaluating: 100%|██████████| 251/251 [00:13<00:00, 18.93it/s]
+
+Epoch 31/50
+  Train Loss: 0.8429 | Train Acc: 79.02%
+  Val Loss: 0.8694 | Val Acc: 78.08%
+  Mean Alpha: 0.9671 | Mean Density: 0.0391
+  ✓ New best model saved! (Val Acc: 78.08%)
+Epoch 32/50: 100%|██████████| 2253/2253 [01:57<00:00, 19.11it/s, loss=0.8387, acc=79.06%]
+Evaluating: 100%|██████████| 251/251 [00:13<00:00, 19.07it/s]
+
+Epoch 32/50
+  Train Loss: 0.8387 | Train Acc: 79.06%
+  Val Loss: 0.8660 | Val Acc: 78.11%
+  Mean Alpha: 0.9680 | Mean Density: 0.0391
+  ✓ New best model saved! (Val Acc: 78.11%)
+Epoch 33/50: 100%|██████████| 2253/2253 [01:58<00:00, 19.07it/s, loss=0.8351, acc=79.12%]
+Evaluating: 100%|██████████| 251/251 [00:13<00:00, 18.81it/s]
+
+Epoch 33/50
+  Train Loss: 0.8351 | Train Acc: 79.12%
+  Val Loss: 0.8629 | Val Acc: 78.18%
+  Mean Alpha: 0.9687 | Mean Density: 0.0391
+  ✓ New best model saved! (Val Acc: 78.18%)
+Epoch 34/50: 100%|██████████| 2253/2253 [01:57<00:00, 19.14it/s, loss=0.8319, acc=79.16%]
+Evaluating: 100%|██████████| 251/251 [00:13<00:00, 18.85it/s]
+
+Epoch 34/50
+  Train Loss: 0.8320 | Train Acc: 79.16%
+  Val Loss: 0.8602 | Val Acc: 78.21%
+  Mean Alpha: 0.9694 | Mean Density: 0.0391
+  ✓ New best model saved! (Val Acc: 78.21%)
+Epoch 35/50: 100%|██████████| 2253/2253 [01:57<00:00, 19.21it/s, loss=0.8290, acc=79.21%]
+Evaluating: 100%|██████████| 251/251 [00:13<00:00, 18.74it/s]
+
+Epoch 35/50
+  Train Loss: 0.8289 | Train Acc: 79.21%
+  Val Loss: 0.8578 | Val Acc: 78.24%
+  Mean Alpha: 0.9699 | Mean Density: 0.0391
+  ✓ New best model saved! (Val Acc: 78.24%)
+Epoch 36/50: 100%|██████████| 2253/2253 [01:57<00:00, 19.16it/s, loss=0.8265, acc=79.25%]
+Evaluating: 100%|██████████| 251/251 [00:13<00:00, 18.69it/s]
+
+Epoch 36/50
+  Train Loss: 0.8265 | Train Acc: 79.25%
+  Val Loss: 0.8558 | Val Acc: 78.26%
+  Mean Alpha: 0.9704 | Mean Density: 0.0391
+  ✓ New best model saved! (Val Acc: 78.26%)
+Epoch 37/50: 100%|██████████| 2253/2253 [01:57<00:00, 19.19it/s, loss=0.8243, acc=79.28%]
+Evaluating: 100%|██████████| 251/251 [00:13<00:00, 18.76it/s]
+
+Epoch 37/50
+  Train Loss: 0.8243 | Train Acc: 79.28%
+  Val Loss: 0.8540 | Val Acc: 78.28%
+  Mean Alpha: 0.9709 | Mean Density: 0.0391
+  ✓ New best model saved! (Val Acc: 78.28%)
+Epoch 38/50: 100%|██████████| 2253/2253 [01:57<00:00, 19.18it/s, loss=0.8224, acc=79.31%]
+Evaluating: 100%|██████████| 251/251 [00:13<00:00, 18.97it/s]
+
+Epoch 38/50
+  Train Loss: 0.8224 | Train Acc: 79.31%
+  Val Loss: 0.8525 | Val Acc: 78.30%
+  Mean Alpha: 0.9712 | Mean Density: 0.0391
+  ✓ New best model saved! (Val Acc: 78.30%)
+Epoch 39/50: 100%|██████████| 2253/2253 [01:57<00:00, 19.19it/s, loss=0.8208, acc=79.33%]
+Evaluating: 100%|██████████| 251/251 [00:13<00:00, 18.89it/s]
+
+Epoch 39/50
+  Train Loss: 0.8207 | Train Acc: 79.33%
+  Val Loss: 0.8512 | Val Acc: 78.32%
+  Mean Alpha: 0.9715 | Mean Density: 0.0391
+  ✓ New best model saved! (Val Acc: 78.32%)
+Epoch 40/50: 100%|██████████| 2253/2253 [01:57<00:00, 19.13it/s, loss=0.8194, acc=79.35%]
+Evaluating: 100%|██████████| 251/251 [00:13<00:00, 18.91it/s]
+
+Epoch 40/50
+  Train Loss: 0.8194 | Train Acc: 79.35%
+  Val Loss: 0.8501 | Val Acc: 78.33%
+  Mean Alpha: 0.9718 | Mean Density: 0.0391
+  ✓ New best model saved! (Val Acc: 78.33%)
+Epoch 41/50: 100%|██████████| 2253/2253 [01:57<00:00, 19.14it/s, loss=0.8184, acc=79.37%]
+Evaluating: 100%|██████████| 251/251 [00:13<00:00, 18.88it/s]
+
+Epoch 41/50
+  Train Loss: 0.8184 | Train Acc: 79.37%
+  Val Loss: 0.8492 | Val Acc: 78.35%
+  Mean Alpha: 0.9720 | Mean Density: 0.0391
+  ✓ New best model saved! (Val Acc: 78.35%)
+Epoch 42/50: 100%|██████████| 2253/2253 [01:57<00:00, 19.22it/s, loss=0.8174, acc=79.39%]
+Evaluating: 100%|██████████| 251/251 [00:13<00:00, 18.50it/s]
+
+Epoch 42/50
+  Train Loss: 0.8174 | Train Acc: 79.39%
+  Val Loss: 0.8485 | Val Acc: 78.36%
+  Mean Alpha: 0.9722 | Mean Density: 0.0391
+  ✓ New best model saved! (Val Acc: 78.36%)
+Epoch 43/50:  96%|█████████▌| 2158/2253 [01:53<00:05, 17.16it/s, loss=0.8170, acc=79.38%]