Fix the chat box being off-screen, the scroll traps, and a sanitiser bug eating hyphens

Hi — these are fixes on top of your LangChain/RAG work (`9c653b9`), found while
reviewing this Space. Nothing of yours is reverted: `langchain_llm.py`, the
semantic search and the prompt changes all stay as you wrote them.

**One bug is live right now.** In `engine._CJK_RE`, the private-use-area range
`'\ue000-\uf8ff'` was flattened to a bare `'-'` somewhere in the web editor. The
class no longer strips PUA characters, and it *does* strip a literal ASCII
hyphen from every model reply — `"التسجيل المباشر - المفاضلة"` comes back
without its dash. Restored.

**`build_llm` would break the Streamlit build.** It imports `llm.langchain_llm`
unconditionally, but `requirements-deploy.txt` does not carry LangChain, so
every chat answer there would silently drop to the deterministic fallback.
LangChain is added to that manifest, and an `ImportError` now falls back to the
provider SDKs — the behaviour `requirements.txt` already describes in its own
comment ("cloud_llm.py is kept only as a documented fallback").

**Four tests were patching a dead path.** `test_build_llm_*` still monkeypatched
`llm.cloud_llm.CloudLLM`, which `build_llm` no longer calls. They patch the
LangChain path now; the contract under test is unchanged, since
`langchain_llm.PROVIDER_PRECEDENCE` is the same list. 148 tests pass with
LangChain installed and without it.

**The chat box was unreachable.** It is the last child of `#kalim-main`, so
`position: sticky; bottom: 3rem` had no room to lift it — measured, it sat
748px below the fold on a 1440x900 desktop and 873px below on a 390px phone. A
student saw ten result cards and no way to ask anything. It is `fixed` now,
tracking the main column. The footer also wraps to two lines at 390px (54px),
while the offsets guarding it were a flat 3rem and, on phones, 1.2rem — less
than the thing they were clearing. A small observer measures the footer, the
column and the composer into CSS variables instead of hardcoding them.

**Two scroll traps.** The conversation auto-scrolled to the bottom on any
mutation, so scrolling up to re-read something pulled you straight back down;
it now sticks only when you are already at the bottom, and a question you typed
re-sticks it. And the pane carried `overscroll-behavior: contain` while being
sized against the whole window, so on anything shorter than 900px the first
wizard question sat below the fold with no way to scroll to it — the pointer
was over a pane that would not chain its scroll.

**The shortlist reads in order.** It was a `gr.render` panel pinned below the
chat, so an answer to a question asked after the list appeared *above* the
list. Each batch is posted as a chat message of `<details>` cards now, so
"show more" appends underneath and so does every later answer. Opening a card
is still a client-side toggle. This needs the chatbot to render HTML, so
escaping moved into `_bot`/`_user` — model replies are escaped, only markup the
file writes is live.

**Startup.** The semantic index was built at import, before `gr.Blocks` was
defined, so the Space served nothing at all until the sentence-transformers
model had downloaded and all 95 majors were embedded — on every ZeroGPU
restart. It builds on a background thread now; the chat works without it.

`CLAUDE.md` records the layout constraints, so the next editing session does
not undo the measured offsets by replacing them with rem values again.

Happy to split this into smaller PRs if you would rather take it in pieces.

CLAUDE.md

@@ -86,6 +86,17 @@ twice, so a client rewording would have landed in one build only.
 option lists come from `SECONDARY_TRACK_NAMES` / `TEACHING_LANGUAGES` for the
 same reason.
 
+In the Gradio build the shortlist is **part of the conversation**, not a panel
+under it: `_cards_message` renders each batch of results as `<details>` cards
+posted as a chat message, so a question asked after the list gets its answer
+*below* the list. It used to be a `gr.render` panel pinned beneath the chat,
+which meant every later answer appeared above the list the student had just
+been given. Two consequences to keep in mind: the chatbot runs with
+`sanitize_html=False`, so `_bot`/`_user` escape everything that is not markup
+this file wrote (model replies above all), and Gradio groups consecutive
+assistant messages into one bubble, so a card message cannot be styled as if
+it owned its bubble — the cards take the surface tint instead.
+
 `engine.free_chat_reply` returns a `ChatTurn(reply, visible)`: the shells store
 `turn.visible` as the new on-screen window and must never re-derive the paging
 decision themselves. The Gradio `chat()` handler used to re-run the
@@ -141,6 +152,24 @@ opening `<div>` never wraps the widgets that follow. Use
 until the next rerun.** The welcome message is seeded in `init_state()` for this
 reason.
 
+## Gradio layout constraints
+
+Three sizes on that page are not knowable to CSS, and every hardcoded offset
+that assumed them was wrong somewhere: the footer wraps to two lines below
+~700px, the main column shifts when the sidebar opens, and the composer does
+not exist until results do. A layout observer in `FORCE_LIGHT` measures them
+into `--kalim-footer-h`, `--kalim-dock-left/-width/-h` and `--kalim-chat-max`.
+Do not replace those with rem values.
+
+The composer is `position: fixed`, not `sticky`. It was sticky, which is a
+no-op for it: the textbox is the last child of `#kalim-main`, so its flow
+position already sits at the bottom of its containing block and sticky has no
+room to lift it — measured, it sat 748px below the fold on a 1440x900 desktop.
+
+A `gr.HTML` whose content is `position: fixed` still leaves its Gradio wrappers
+in flow. The footer's wrappers added ~52px of empty page, which is why they are
+collapsed explicitly.
+
 ## RTL
 
 The app shell stays LTR (`[data-testid="stAppViewContainer"] { direction: ltr }`)

app_gradio.py

@@ -21,6 +21,9 @@ from pathlib import Path
 from functools import lru_cache
 import os
 import sys
+import threading
+import re
+from html import escape
 
 # Ensure project root and src directory are in Python path
 _PROJECT_ROOT = str(Path(__file__).parent.resolve())
@@ -73,16 +76,28 @@ from core import engine
 # `python -m src.rag.vectorstore` manually, so this runs it once per restart
 # instead. Guarded so a missing optional dependency degrades to no semantic
 # search rather than crashing the whole app.
-try:
-    from rag.vectorstore import get_vectorstore
-
-    _vs = get_vectorstore()
-    if _vs.collection.count() == 0:
-        print("Building semantic search index (first run)...")
-        _vs.build_index()
-        print(f"Index built: {_vs.collection.count()} documents")
-except Exception as _e:
-    print(f"Semantic index not built (optional feature disabled): {_e}")
+#
+# On a background thread, not at import: building it inline downloads the
+# sentence-transformers model and embeds all 95 majors *before* gr.Blocks is
+# even defined, so the Space served nothing at all until it finished — the
+# whole cold start, on every ZeroGPU restart. The chat works without it
+# (engine.semantic_majors_mentioned only runs when a question names no major
+# literally, and returns nothing while the collection is empty), so the index
+# can finish after the UI is already up.
+def _build_semantic_index():
+    try:
+        from rag.vectorstore import get_vectorstore
+
+        vs = get_vectorstore()
+        if vs.collection.count() == 0:
+            print("Building semantic search index (first run)...")
+            vs.build_index()
+            print(f"Index built: {vs.collection.count()} documents")
+    except Exception as exc:
+        print(f"Semantic index not built (optional feature disabled): {exc}")
+
+
+threading.Thread(target=_build_semantic_index, name="kalim-index", daemon=True).start()
 
 NO_TRACK = "غير محدد"
 LANG_FLAGS = {"العربية": "🇱🇧", "الانكليزية": "🇬🇧", "الفرنسية": "🇫🇷"}
@@ -113,11 +128,86 @@ def _initial_state() -> dict:
 
 
 def _initial_history() -> list:
-    return [{'role': 'assistant', 'content': WELCOME_MESSAGE}]
+    return [{'role': 'assistant', 'content': _as_text(WELCOME_MESSAGE)}]
+
+
+# gr.Chatbot renders raw HTML for us (sanitize_html=False), which makes
+# escaping this shell's job. Everything that is not markup written in this file
+# goes through here first: dataset text, the engine's wording, and above all
+# model replies, which are the one input a student could steer. Markdown is
+# unaffected — its syntax does not use these three characters.
+def _as_text(content: str) -> str:
+    return escape(str(content), quote=False)
 
 
 def _bot(history, text):
-    history.append({'role': 'assistant', 'content': text})
+    history.append({'role': 'assistant', 'content': _as_text(text)})
+    return history
+
+
+# --- The shortlist, as part of the conversation ------------------------------
+# The cards used to be a gr.render panel pinned below the chat, which meant a
+# later answer appeared *above* the list the student had just been given — the
+# newest thing on screen was not the newest thing said. They are chat content
+# now: each batch is posted as its own message, so the conversation reads in
+# the order it happened and anything asked afterwards lands underneath.
+#
+# <details>/<summary> rather than gr.Accordion: only a component can go in the
+# component tree, and this has to live inside a message. Collapsing is native
+# to the element, so it stays a click with no round trip, exactly as before.
+# gr.Chatbot(allow_tags=...) lets these two tags through its sanitiser; every
+# other tag, including anything a model might emit, is still stripped.
+_MD_BOLD = re.compile(r'\*\*(.+?)\*\*', re.S)
+_MD_ITALIC = re.compile(r'(?<!\*)\*([^*]+?)\*(?!\*)', re.S)
+
+
+def _mini_markdown(text: str) -> str:
+    """The little markdown format_major_details emits, as HTML.
+
+    Only bold, italics and paragraph breaks — everything is escaped first, so
+    dataset text cannot inject markup.
+    """
+    out = []
+    for para in escape(text).split("\n\n"):
+        para = para.strip()
+        if not para:
+            continue
+        para = _MD_BOLD.sub(r'<strong>\1</strong>', para)
+        para = _MD_ITALIC.sub(r'<em>\1</em>', para)
+        out.append(f'<p>{para}</p>')
+    return "".join(out)
+
+
+def _cards_message(results, start: int = 0) -> str:
+    """Collapsible cards for results[start:], numbered from their real rank."""
+    cards = []
+    for position, result in enumerate(results[start:], start + 1):
+        major = result.get('major', {})
+        header = (
+            f"{position}. {major.get('name_ar', '')} — "
+            f"{major.get('faculty_ar', '')}  ·  "
+            f"{result.get('holland_score', 0)}% مطابقة"
+        )
+        cards.append(
+            '<details class="kalim-major-card">'
+            f'<summary>{escape(header)}</summary>'
+            f'<div class="kalim-major-details">'
+            f'{_mini_markdown(engine.format_major_details(major))}</div>'
+            '</details>'
+        )
+    return '<div class="kalim-cards">' + "".join(cards) + '</div>'
+
+
+def _bot_cards(history, results, start: int = 0):
+    """Post one batch of cards, if there is anything new to post.
+
+    Appends the markup directly rather than through _bot: this is the one
+    message whose HTML is meant to be live, and its own text was escaped
+    field by field in _cards_message.
+    """
+    if len(results) > start:
+        history.append({'role': 'assistant',
+                        'content': _cards_message(results, start)})
     return history
 
 
@@ -158,9 +248,11 @@ def apply_search(holland_sel, track_sel, lang_sel, gov_sel, history, state):
         "، ".join(govs) or None,
     ]))
     history = _initial_history()
-    history.append({'role': 'user', 'content': f"بحث جديد: {summary}"})
+    _user(history, f"بحث جديد: {summary}")
     reply = engine.results_intro(results, codes, interactive_cards=False)
-    return (_bot(history, reply), state, *_vis('results'))
+    _bot(history, reply)
+    _bot_cards(history, state['visible'])
+    return (history, state, *_vis('results'))
 
 
 def show_more(history, state):
@@ -173,14 +265,15 @@ def show_more(history, state):
 
     nxt = results[len(visible):len(visible) + engine.PAGE_SIZE]
     shown = len(visible) + len(nxt)
-    # The cards below re-render from `state`, so the new majors simply appear;
-    # the chat only narrates what happened.
     reply = (
         f"أضفت التخصصات من {len(visible) + 1} إلى {shown} "
         f"(من أصل {len(results)}) إلى القائمة في الأسفل."
     )
+    _bot(history, reply)
+    # Only the new batch, numbered from where the last one stopped.
+    _bot_cards(history, results[:shown], start=len(visible))
     state = {**state, 'visible': results[:shown]}
-    return _bot(history, reply), state
+    return history, state
 
 
 def compare(history, state):
@@ -202,7 +295,7 @@ def chat(message, history, state):
     if not message:
         return "", history, state
 
-    history.append({'role': 'user', 'content': message})
+    _user(history, message)
     visible, results = state['visible'], state['results']
     # Past the per-session budget the answer still comes, from the data rather
     # than the model. Never an error — see engine.MAX_LLM_TURNS_PER_SESSION.
@@ -216,8 +309,13 @@ def chat(message, history, state):
         llm=None if over_budget else get_llm(),
     )
     # The engine owns the paging decision; the shell only stores its result.
+    _bot(history, turn.reply)
+    # A paged-in answer widens the window — those majors are new to the
+    # student, so they follow the reply as their own batch of cards.
+    if len(turn.visible) > len(visible):
+        _bot_cards(history, turn.visible, start=len(visible))
     state = {**state, 'visible': turn.visible}
-    return "", _bot(history, turn.reply), state
+    return "", history, state
 
 
 # ---------------------------------------------------------------------------
@@ -262,7 +360,7 @@ def _sync(state):
 
 
 def _user(history, text):
-    history.append({'role': 'user', 'content': text})
+    history.append({'role': 'user', 'content': _as_text(text)})
     return history
 
 
@@ -353,6 +451,7 @@ def _finish_wizard(history, state, govs, dists):
              'results': results, 'visible': results[:engine.PAGE_SIZE]}
     reply = engine.results_intro(results, state['holland_codes'], interactive_cards=False)
     _bot(history, reply)
+    _bot_cards(history, state['visible'])
     return (history, state, *_vis('results'), *_sync(state))
 
 
@@ -407,16 +506,28 @@ CSS = """
 }
 
 body {
-    /* Atmospheric page ground: airy blue-white with a navy glow (top), a warm
-       cedar glow (bottom) and a faint dot grid — cards stay white on top.
-       CSS-only: the HF Space remote rejects plain-git binary files. */
     background-color: #fbfcfe !important;
+    color: var(--text);
+}
+/* Atmospheric page ground: airy blue-white with a navy glow (top), a warm
+   cedar glow (bottom) and a faint dot grid — cards stay white on top.
+   CSS-only: the HF Space remote rejects plain-git binary files.
+   Painted on a fixed pseudo-element rather than as three
+   `background-attachment: fixed` layers on <body>: attachment:fixed forces the
+   browser to repaint the gradients and the dot tile against new scroll offsets
+   on every frame, which is the one thing on this page that costs real work
+   while scrolling. As its own composited layer it is painted once. */
+body::before {
+    content: "";
+    position: fixed;
+    inset: 0;
+    z-index: -1;
+    pointer-events: none;
+    background-color: #fbfcfe;
     background-image:
         radial-gradient(1200px 600px at 85% -10%, rgba(42, 82, 152, 0.12), transparent 60%),
         radial-gradient(1000px 520px at -10% 108%, rgba(179, 84, 30, 0.08), transparent 55%),
-        url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' width='26' height='26'%3E%3Ccircle cx='1.4' cy='1.4' r='1.1' fill='%232a5298' fill-opacity='0.06'/%3E%3C/svg%3E") !important;
-    background-attachment: fixed, fixed, fixed;
-    color: var(--text);
+        url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' width='26' height='26'%3E%3Ccircle cx='1.4' cy='1.4' r='1.1' fill='%232a5298' fill-opacity='0.06'/%3E%3C/svg%3E");
 }
 /* <gradio-app> and the container both paint white by default, hiding the
    body's texture — make the whole stack transparent down to body. */
@@ -430,6 +541,35 @@ gradio-app, .gradio-container {
 }
 .gradio-container *:not([class*="icon"]) { font-family: var(--kalim-font) !important; }
 
+/* ===== Scrollbars =====
+   The app legitimately has more than one scroll region (the page, the
+   conversation, the sidebar, a wide result table), and each arrived with the
+   platform's default chrome — a stack of grey rails that read as clutter.
+   One quiet treatment for all of them: thin, no track, thumb only. */
+* {
+    scrollbar-width: thin;
+    scrollbar-color: var(--line-strong) transparent;
+}
+*::-webkit-scrollbar { width: 9px; height: 9px; }
+*::-webkit-scrollbar-track { background: transparent; }
+*::-webkit-scrollbar-corner { background: transparent; }
+*::-webkit-scrollbar-thumb {
+    background: var(--line-strong);
+    background-clip: content-box;
+    border: 2px solid transparent;
+    border-radius: 999px;
+}
+*::-webkit-scrollbar-thumb:hover {
+    background: var(--ink-300);
+    background-clip: content-box;
+}
+
+/* Gradio wraps several blocks in scroll containers that never actually
+   overflow — they contribute an inert rail and nothing else. */
+#kalim-main .form, #kalim-main > .block, .kalim-major-details {
+    overflow: visible !important;
+}
+
 /* Platform chrome: Gradio's own footer, and the owner pill Hugging Face
    injects into this page's body on direct .hf.space visits. */
 footer { display: none !important; }
@@ -617,34 +757,81 @@ ul.options .item, ul.options li { text-align: right; }
     outline-offset: 2px !important;
 }
 
-/* Result cards — the browsable shortlist under the conversation. Each is a
-   collapsed accordion; opening one is a client-side toggle, so details appear
-   instantly the way the Streamlit expanders do. */
+/* Result cards — the shortlist, posted into the conversation as its own
+   message so anything asked afterwards appears below it. Each card is a
+   native <details>, so opening one is a client-side toggle with no round
+   trip, the way the Streamlit expanders behave. */
+.kalim-cards { direction: rtl; text-align: right; }
 .kalim-major-card {
-    border: 1px solid var(--line) !important;
-    border-radius: var(--radius) !important;
-    background: var(--bg) !important;
-    box-shadow: var(--kalim-shadow) !important;
-    margin: 0 0 0.5rem 0 !important;
+    border: 1px solid var(--line);
+    border-radius: var(--radius);
+    background: var(--bg);
+    box-shadow: var(--kalim-shadow);
+    margin: 0 0 0.5rem 0;
     overflow: hidden;
+    transition: border-color .18s ease, box-shadow .18s ease;
 }
-.kalim-major-card > .label-wrap,
-.kalim-major-card button.label-wrap {
-    direction: rtl !important;
-    text-align: right !important;
-    font-weight: 600 !important;
-    color: var(--ink-700) !important;
-    padding: 0.7rem 0.9rem !important;
+.kalim-major-card:last-child { margin-bottom: 0; }
+.kalim-major-card > summary {
+    direction: rtl;
+    text-align: right;
+    font-weight: 600;
+    color: var(--ink-700);
+    padding: 0.7rem 0.9rem;
+    cursor: pointer;
+    list-style: none;
+    display: flex;
+    align-items: center;
+    gap: 0.6rem;
+}
+.kalim-major-card > summary::-webkit-details-marker { display: none; }
+/* A chevron of our own: the platform marker points sideways when closed,
+   which reads as "next" rather than "expand". */
+.kalim-major-card > summary::after {
+    content: "";
+    margin-inline-start: auto;
+    width: 0.5rem; height: 0.5rem;
+    border-right: 2px solid var(--text-faint);
+    border-bottom: 2px solid var(--text-faint);
+    transform: rotate(45deg) translateY(-2px);
+    transition: transform .18s ease;
+    flex-shrink: 0;
 }
+.kalim-major-card[open] > summary::after { transform: rotate(-135deg) translateY(-2px); }
 .kalim-major-card:hover {
-    border-color: var(--line-strong) !important;
+    border-color: var(--line-strong);
+    box-shadow: 0 2px 6px rgba(16,36,63,.06), 0 8px 22px rgba(16,36,63,.08);
+}
+.kalim-major-card > summary:focus-visible {
+    outline: 2px solid var(--cedar);
+    outline-offset: -2px;
 }
 .kalim-major-details {
-    direction: rtl !important;
-    text-align: right !important;
-    padding: 0 0.9rem 0.6rem !important;
+    direction: rtl;
+    text-align: right;
+    padding: 0 0.9rem 0.75rem;
+    border-top: 1px solid var(--line);
+    padding-top: 0.6rem;
 }
 .kalim-major-details p { margin: 0.35rem 0 !important; }
+.kalim-major-details strong { color: var(--ink-700); }
+
+/* The shortlist sits inside Kalim's turn rather than replacing it: Gradio
+   groups consecutive assistant messages into a single bubble, so stripping
+   the bubble for a card message stripped it from the narration beside it too.
+   The cards take the surface tint instead, which separates them from the
+   white bubble they sit in and ties the list to the message that introduced
+   it. */
+#kalim-chat .message .kalim-major-card { background: var(--surface); }
+#kalim-chat .message .kalim-major-card[open],
+#kalim-chat .message .kalim-major-card:hover { background: var(--bg); }
+#kalim-chat .kalim-cards { margin: 0.5rem 0 0.15rem; }
+/* A turn carrying the shortlist needs the column, not a chat bubble's width. */
+#kalim-chat .message-row:has(.kalim-cards),
+#kalim-chat .message.bot:has(.kalim-cards) {
+    max-width: 100% !important;
+    width: 100% !important;
+}
 
 /* ===== Chat ===== */
 /* Styled as a modern messenger: centered column, avatars, distinct bubbles,
@@ -653,6 +840,23 @@ ul.options .item, ul.options li { text-align: right; }
    further away with every answer, so the shortlist appeared to jump after each
    new reply instead of staying put. */
 #kalim-main { max-width: 880px; margin: 0 auto; width: 100%; }
+/* Before a search there is one short welcome card and a button, which left
+   roughly 370px of empty page under them on a 900px window — the app read as
+   unfinished rather than calm. Until results exist the column fills the free
+   height and centres what is in it; once the shortlist appears the class goes
+   away and normal top-down flow returns. The class is set by the layout
+   observer alongside the measurements it already takes. */
+body:not(.kalim-has-results) .gradio-container { min-height: 100vh; }
+/* flex-grow only reaches the column if every wrapper between it and the
+   container grows too — Gradio nests a couple of unnamed divs in there. */
+body:not(.kalim-has-results) .gradio-container > div,
+body:not(.kalim-has-results) div:has(> #kalim-main) {
+    flex: 1 1 auto;
+}
+body:not(.kalim-has-results) #kalim-main {
+    flex: 1 1 auto;
+    justify-content: center;
+}
 #kalim-chat {
     border: none !important;
     box-shadow: none !important;
@@ -669,9 +873,25 @@ ul.options .item, ul.options li { text-align: right; }
 #kalim-chat .bubble-wrap {
     background: transparent !important;
     height: auto !important;
-    max-height: 58vh !important;
+    /* Was a flat 58vh (46vh on phones), which left ~370px of dead space under
+       the welcome screen and squeezed the conversation on short windows. The
+       observer measures what is actually free between the chat's top and the
+       two docked layers. */
+    max-height: var(--kalim-chat-max, 58vh) !important;
     overflow-y: auto !important;
-    overscroll-behavior: contain;
+    /* Deliberately NOT `overscroll-behavior: contain`. Containing it means a
+       wheel over the conversation never reaches the page, and the conversation
+       covers most of the window — so on a short window the wizard step below
+       it was unreachable by scrolling at all. Chaining at the boundary is the
+       behaviour people expect from a pane inside a page. */
+    overscroll-behavior-y: auto;
+    scrollbar-gutter: stable;
+}
+/* Scrollback affordance: without it the pane looks like a conversation that
+   begins mid-sentence. The class is toggled by the head script on scroll. */
+#kalim-chat.kalim-scrolled .bubble-wrap {
+    -webkit-mask-image: linear-gradient(to bottom, transparent 0, #000 26px);
+            mask-image: linear-gradient(to bottom, transparent 0, #000 26px);
 }
 #kalim-chat .message-wrap { gap: 1.05rem !important; }
 /* No chat chrome: the clear-history icon is the last one buttons=[] leaves. */
@@ -775,9 +995,20 @@ ul.options .item, ul.options li { text-align: right; }
     border: none !important;
     box-shadow: none !important;
 }
+/* Docked to the viewport, over the main column. It used to be `position:
+   sticky; bottom: 3rem`, which is a no-op here: the textbox is the LAST child
+   of #kalim-main, so its flow position already sits at the bottom of its
+   containing block and sticky has no room to lift it. Measured, it sat 748px
+   below the fold on a 1440x900 desktop and 873px below it on a 390px phone —
+   a student saw ten result cards and no way to ask anything. `fixed` plus the
+   measured column geometry below keeps it on screen at every width.
+   --kalim-dock-* and --kalim-footer-h are written by the layout observer in
+   the head script; the fallbacks keep it usable if that script never runs. */
 #kalim-input {
-    position: sticky;
-    bottom: 3rem;
+    position: fixed;
+    bottom: calc(var(--kalim-footer-h, 2.4rem) + 0.5rem);
+    left: var(--kalim-dock-left, 0px);
+    width: var(--kalim-dock-width, min(880px, calc(100% - 2rem)));
     z-index: 900;
     background: var(--bg);
     border: 1px solid var(--line-strong) !important;
@@ -785,6 +1016,20 @@ ul.options .item, ul.options li { text-align: right; }
     box-shadow: 0 6px 24px rgba(16, 36, 63, 0.10);
     padding: 0.15rem 0.4rem !important;
 }
+/* The cards scroll underneath the pill; fade the ground out around it so they
+   do not appear to run into its rounded edge. */
+#kalim-input::before {
+    content: "";
+    position: absolute;
+    left: -1rem; right: -1rem;
+    bottom: calc(-1 * var(--kalim-footer-h, 2.4rem) - 0.5rem);
+    top: -0.9rem;
+    z-index: -1;
+    pointer-events: none;
+    background: linear-gradient(to bottom, rgba(251, 252, 254, 0) 0%,
+                                           rgba(251, 252, 254, 0.92) 45%,
+                                           rgba(251, 252, 254, 1) 100%);
+}
 #kalim-input textarea {
     direction: rtl !important; text-align: right !important;
     font-size: 16px !important;   /* keeps iOS Safari from zooming on focus */
@@ -813,6 +1058,24 @@ ul.options .item, ul.options li { text-align: right; }
 #kalim-input button * { color: #fff !important; fill: #fff !important; }
 
 /* ===== Footer — mirrors .kalim-footer: fixed to the bottom edge ===== */
+/* The strip itself is out of flow, but the wrappers Gradio puts around a
+   gr.HTML are not: their padding and the column's gap added ~52px of empty
+   page under the last element, which is what put a scrollbar on a welcome
+   screen that otherwise fits the window exactly. Collapse them. */
+.gradio-container .block:has(#kalim-footer),
+.gradio-container .html-container:has(#kalim-footer),
+.gradio-container .prose:has(#kalim-footer) {
+    height: 0 !important;
+    min-height: 0 !important;
+    padding: 0 !important;
+    margin: 0 !important;
+    border: none !important;
+    overflow: visible !important;
+}
+/* Zero height is not enough: a flex item still earns the column's gap, which
+   left 32px of page below the last element. Taking it out of flow entirely
+   removes the gap with it, and costs nothing — the strip inside is fixed. */
+.gradio-container .block:has(#kalim-footer) { position: absolute !important; }
 #kalim-footer {
     position: fixed;
     bottom: 0; left: 0; right: 0;
@@ -825,8 +1088,14 @@ ul.options .item, ul.options li { text-align: right; }
     margin: 0;
     z-index: 998;
 }
-/* Keep the last controls clear of the fixed strip. */
-.gradio-container { padding-bottom: 3rem !important; }
+/* Keep the last card clear of BOTH fixed layers — the footer and the docked
+   composer. 3rem was a guess against a one-line footer; the footer wraps to
+   two lines at 390px (54px measured), so the strip covered the last result
+   card. Both heights are measured at runtime instead. */
+.gradio-container {
+    padding-bottom: calc(var(--kalim-footer-h, 2.4rem)
+                         + var(--kalim-dock-h, 0px) + 1.75rem) !important;
+}
 
 @media (max-width: 1024px) {
     #kalim-header h1 { font-size: 1.45rem; }
@@ -842,8 +1111,8 @@ ul.options .item, ul.options li { text-align: right; }
     #kalim-chat .message-row .message,
     #kalim-chat .message { max-width: 100% !important; }
     #kalim-chat .message-row { gap: 0.4rem !important; }
-    /* Leave room for the cards and the input on a short screen. */
-    #kalim-chat .bubble-wrap { max-height: 46vh !important; }
+    /* Height comes from --kalim-chat-max, which is measured against this
+       viewport too — no separate phone fraction to keep in sync. */
 
     /* Hide Kalim's avatar on phones: the 35px circle plus its gutter costs
        more width than it communicates, and Kalim's turns are the long ones
@@ -861,8 +1130,12 @@ ul.options .item, ul.options li { text-align: right; }
         margin-right: var(--spacing-xl) !important;
     }
     #kalim-actions { flex-wrap: wrap; }
-    #kalim-input { bottom: 1.2rem; }
-    .gradio-container { padding: 0.5rem 0.5rem 3rem !important; }
+    /* 1.2rem was less than the two-line footer this width produces, so the
+       strip sat on top of the composer. The measured offset handles it. */
+    .gradio-container {
+        padding: 0.5rem 0.5rem calc(var(--kalim-footer-h, 3.4rem)
+                                    + var(--kalim-dock-h, 0px) + 1.5rem) !important;
+    }
 }
 """
 
@@ -908,27 +1181,216 @@ FORCE_LIGHT = """
 })();
 </script>
 <script>
-/* Follow the conversation: whenever a new message lands (every wizard click
-   posts one), scroll to the newest content *inside* the chat. Scrolling the
-   whole page instead would drag the result cards below the fold on every
-   answer — the conversation moves, the shortlist stays where it is. */
+/* Layout observer — writes the geometry the CSS cannot know.
+ *
+ * Three things on this page have no fixed size: the footer wraps to two lines
+ * below ~700px, the main column shifts left when the sidebar opens, and the
+ * composer is absent until results exist. Every offset that used to be a
+ * hardcoded rem value was wrong in at least one of those states — the footer
+ * covered the last result card at 390px, and the composer's bottom offset was
+ * smaller than the footer it was supposed to clear. These are measured
+ * instead, so they are right at every width, zoom level and sidebar state.
+ *
+ *   --kalim-footer-h   height of the fixed footer strip
+ *   --kalim-dock-left  left edge of the main column (the composer tracks it)
+ *   --kalim-dock-width width of the main column
+ *   --kalim-dock-h     height of the docked composer, 0 while it is hidden
+ *   --kalim-chat-max   how tall the conversation may grow before it scrolls
+ */
 (function () {
+  const root = document.documentElement;
+  const px = n => Math.round(n) + 'px';
+  let queued = false;
+
+  const set = (name, value) => {
+    if (root.style.getPropertyValue(name) !== value) {
+      root.style.setProperty(name, value);
+    }
+  };
+
+  const measure = () => {
+    queued = false;
+    const footer = document.querySelector('#kalim-footer');
+    const main = document.querySelector('#kalim-main');
+    const chat = document.querySelector('#kalim-chat');
+    const dock = document.querySelector('#kalim-input');
+
+    const footerH = footer ? footer.offsetHeight : 0;
+    set('--kalim-footer-h', px(footerH));
+
+    if (main) {
+      const r = main.getBoundingClientRect();
+      set('--kalim-dock-left', px(r.left));
+      set('--kalim-dock-width', px(r.width));
+    }
+
+    // offsetParent is null while Gradio keeps the textbox display:none.
+    const dockH = dock && dock.offsetParent !== null ? dock.offsetHeight : 0;
+    set('--kalim-dock-h', px(dockH));
+
+    // The composer only exists once a search has produced results, which makes
+    // it the signal for "past the welcome step". The shortlist cards used to
+    // be that signal, but they are inside the conversation now.
+    document.body.classList.toggle('kalim-has-results', dockH > 0);
+
+    if (chat) {
+      // Where the conversation starts, in document coordinates — stable
+      // regardless of how far the page happens to be scrolled.
+      const top = chat.getBoundingClientRect().top + window.scrollY;
+      // Free height between the top of the conversation and the docked layers.
+      // Filling the window before results exist is done in CSS (flex-grow on
+      // the column) rather than here — every attempt to compute that height
+      // from measurements had to guess at Gradio's own padding and gaps.
+      let free = window.innerHeight - top - footerH - dockH - 28;
+
+      // Whatever sits under the conversation inside the column — the wizard
+      // step, the action chips — has to fit too. Without this the pane took
+      // the whole window and pushed the step below the fold: on a 720px-tall
+      // window the "نعم، أعرف رموزي" button started at 765px, and the pointer
+      // was over the conversation, which does not chain its scroll, so there
+      // was no way to reach the button with a wheel. The controls do not
+      // change height when the pane does, so this cannot oscillate.
+      if (main) {
+        const belowChat = main.getBoundingClientRect().bottom
+                        - chat.getBoundingClientRect().bottom;
+        free -= Math.max(0, belowChat);
+      }
+      set('--kalim-chat-max', px(Math.max(200, free)));
+    }
+  };
+
+  const schedule = () => {
+    if (queued) return;
+    queued = true;
+    requestAnimationFrame(measure);
+  };
+
+  const start = () => {
+    const main = document.querySelector('#kalim-main');
+    const footer = document.querySelector('#kalim-footer');
+    if (!main || !footer) { setTimeout(start, 250); return; }
+    measure();
+    if (window.ResizeObserver) {
+      const ro = new ResizeObserver(schedule);
+      ro.observe(main);
+      ro.observe(footer);
+      ro.observe(document.documentElement);
+    }
+    window.addEventListener('resize', schedule);
+    // The sidebar toggle animates the column, and the wizard shows and hides
+    // the composer — both change the numbers above.
+    new MutationObserver(schedule).observe(document.body,
+      { childList: true, subtree: true, attributes: true,
+        attributeFilter: ['class', 'style'] });
+    // The 0.3s margin transition on the column settles after the mutation.
+    setTimeout(measure, 400);
+  };
+
+  if (document.readyState === 'loading') {
+    document.addEventListener('DOMContentLoaded', start);
+  } else {
+    start();
+  }
+})();
+</script>
+<script>
+/* Mark the conversation as scrolled, so the CSS can fade its top edge. A pane
+   that is scrolled but shows no cue reads as a conversation starting
+   mid-sentence. */
+(function () {
+  const attach = () => {
+    const chat = document.querySelector('#kalim-chat');
+    const scroller = chat && chat.querySelector('.bubble-wrap');
+    if (!scroller) { setTimeout(attach, 300); return; }
+    const update = () => chat.classList.toggle('kalim-scrolled', scroller.scrollTop > 8);
+    scroller.addEventListener('scroll', update, { passive: true });
+    update();
+  };
+  if (document.readyState === 'loading') {
+    document.addEventListener('DOMContentLoaded', attach);
+  } else {
+    attach();
+  }
+})();
+</script>
+<script>
+/* Follow the conversation: when a new message lands (every wizard click posts
+   one), scroll to the newest content *inside* the chat, not the page.
+ *
+ * Only while the student is already at the bottom. Scrolling back up through
+ * the shortlist used to snap straight back down: the observer fires on any
+ * mutation in the pane, and reading the conversation produces plenty of them —
+ * opening a card, Gradio re-rendering rows. Sticking to the bottom is right
+ * when you are watching the conversation arrive and wrong the moment you have
+ * gone looking for something, so it is a mode, and the student sets it by
+ * scrolling.
+ */
+(function () {
+  const NEAR_BOTTOM = 80;   // px of slack, so "almost at the bottom" still sticks
   let pending = null;
+  let stick = true;
+  let selfScrolling = false;
+  let selfScrollTimer = null;
+
+  const watchScrolling = (scroller) => {
+    if (scroller.__kalimWatched) return;
+    scroller.__kalimWatched = true;
+    scroller.addEventListener('scroll', () => {
+      // A smooth scroll of our own fires scroll events all the way down, and
+      // reading those as the student's position turned stick off mid-flight —
+      // the pane stopped short of the newest message instead of reaching it.
+      if (selfScrolling) return;
+      const distance = scroller.scrollHeight - scroller.scrollTop - scroller.clientHeight;
+      stick = distance < NEAR_BOTTOM;
+    }, { passive: true });
+  };
+
+  const scrollToNewest = (scroller) => {
+    selfScrolling = true;
+    clearTimeout(selfScrollTimer);
+    scroller.scrollTo({ top: scroller.scrollHeight, behavior: 'smooth' });
+    // Long enough for the animation to land; a scroll the student makes during
+    // it is missed, which is a far smaller cost than mistaking our own.
+    selfScrollTimer = setTimeout(() => { selfScrolling = false; }, 900);
+  };
+
   const schedule = () => {
     clearTimeout(pending);
     pending = setTimeout(() => {
       const chat = document.querySelector('#kalim-chat');
       const scroller = chat && chat.querySelector('.bubble-wrap');
-      if (scroller) {
-        scroller.scrollTo({ top: scroller.scrollHeight, behavior: 'smooth' });
-      }
+      if (!scroller) return;
+      watchScrolling(scroller);
+      if (!stick) return;
+      scrollToNewest(scroller);
     }, 160);
   };
+
+  // Where the student has scrolled to wins, with exactly one exception: the
+  // question they just typed. Sending it is the one action that is explicitly
+  // "and now show me the answer", and the composer they typed it into is
+  // docked at the bottom of the screen anyway.
+  //
+  // Every other button used to re-stick too — the action chips, the wizard
+  // steps — which meant scrolling to the top and then pressing anything threw
+  // the student straight back to the bottom.
+  const sendFromComposer = (event) =>
+    event.target.closest && event.target.closest('#kalim-input');
+  document.addEventListener('click', (event) => {
+    if (sendFromComposer(event)) stick = true;
+  }, true);
+  document.addEventListener('keydown', (event) => {
+    if (event.key === 'Enter' && sendFromComposer(event)) stick = true;
+  }, true);
+
   const attach = () => {
     const chat = document.querySelector('#kalim-chat');
     if (!chat) { setTimeout(attach, 300); return; }
+    const scroller = chat.querySelector('.bubble-wrap');
+    if (scroller) watchScrolling(scroller);
     new MutationObserver(schedule).observe(chat, { childList: true, subtree: true });
   };
+
   if (document.readyState === 'loading') {
     document.addEventListener('DOMContentLoaded', attach);
   } else {
@@ -999,6 +1461,12 @@ with gr.Blocks(title=PAGE_TITLE) as demo:
             show_label=False, buttons=[],
             avatar_images=("assets/avatar-student.svg", "assets/avatar-kalim.svg"),
             elem_id="kalim-chat",
+            # The shortlist is posted as chat content (see _cards_message), and
+            # <details> is a standard tag, so allow_tags — which only preserves
+            # *custom* tags such as <thinking> — does not cover it. HTML is
+            # rendered instead, and the escaping moves into _bot/_user below so
+            # that only markup this file wrote is ever live.
+            sanitize_html=False,
         )
 
         # Wizard steps — one visible at a time, in place of the chat controls.
@@ -1040,25 +1508,6 @@ with gr.Blocks(title=PAGE_TITLE) as demo:
                 confirm_loc_btn = gr.Button("تأكيد", variant="primary")
                 all_locs_btn = gr.Button("جميع المحافظات")
 
-        # Clickable result cards, the Streamlit build's expander list ported
-        # over: the chat narrates, this panel is the browsable shortlist.
-        # gr.render rebuilds it whenever `state` changes, so "عرض تخصصات
-        # إضافية" simply grows the window and more cards appear — no fixed
-        # pool of components to keep in sync.
-        @gr.render(inputs=state)
-        def render_result_cards(current):
-            visible = (current or {}).get('visible') or []
-            for position, result in enumerate(visible, 1):
-                major = result.get('major', {})
-                header = (
-                    f"{position}. {major.get('name_ar', '')} — "
-                    f"{major.get('faculty_ar', '')}  ·  "
-                    f"{result.get('holland_score', 0)}% مطابقة"
-                )
-                with gr.Accordion(header, open=False, elem_classes="kalim-major-card"):
-                    gr.Markdown(engine.format_major_details(major),
-                                elem_classes="kalim-major-details")
-
         with gr.Row(elem_id="kalim-actions", visible=False) as actions_row:
             more_btn = gr.Button("عرض تخصصات إضافية ⬇️", size="sm")
             compare_btn = gr.Button("📊 قارن التخصصات", size="sm")
@@ -1111,4 +1560,4 @@ with gr.Blocks(title=PAGE_TITLE) as demo:
 if __name__ == "__main__":
     # Gradio 6 takes presentation options at launch time. Hugging Face Spaces
     # runs this file as __main__, so this path serves the Space too.
-    demo.launch(theme=theme, css=CSS, head=FORCE_LIGHT)
+    demo.launch(theme=theme, css=CSS, head=FORCE_LIGHT)

config/prompts.py

@@ -116,4 +116,4 @@ MSG_STATION_4 = (
     "**المحطة الرابعة: الموقع الجغرافي**\n\n"
     "اختر المحافظة أو المحافظات التي تريد ومن ثم اختر القضاء، "
     "لكي أقترح لك فروع الجامعة اللبنانية الأقرب لك."
-)
+)

config/settings.py

@@ -135,7 +135,7 @@ CLOUD_MODELS: Dict[str, str] = {
 }
 
 LLM_TEMPERATURE = 0.7
-LLM_MAX_TOKENS = 4096 
+LLM_MAX_TOKENS = 4096
 LLM_TIMEOUT_SECONDS = 60
 
 # RAG Configuration

requirements-deploy.txt

@@ -9,5 +9,11 @@
 # st.iframe fallback.
 
 streamlit==1.62.0
-groq==1.6.0
 python-dotenv==1.2.3
+
+# LangChain: engine.build_llm goes through llm.langchain_llm, so the Streamlit
+# build needs it too — without it every chat answer silently drops to the
+# deterministic fallback. groq is unpinned for the same reason as in
+# requirements.txt: langchain-groq caps it at <1.0.0.
+langchain-core==0.3.*
+langchain-groq

requirements.txt

@@ -34,4 +34,4 @@ langchain-community==0.3.*
 # which conflicts with gradio==6.25.0's requirement of huggingface-hub>=1.16.
 # Letting pip resolve freely tests whether a newer release has dropped that cap.
 sentence-transformers
-chromadb
+chromadb

src/core/engine.py

@@ -80,12 +80,61 @@ def build_llm(get_secret: Callable[[str], str] = os.getenv):
     provider's SDK directly — this is the thesis's LangChain design
     proposition, implemented in the code path a student's question
     actually travels through.
+
+    LangChain is not installed everywhere this engine runs: requirements.txt
+    (the Hugging Face Space) carries it, requirements-deploy.txt (Streamlit
+    Community Cloud, Docker) does not have to. An ImportError therefore falls
+    back to the provider SDKs in llm.cloud_llm instead of taking the chat
+    down — the behaviour requirements.txt already claims in its comment
+    ("cloud_llm.py is kept only as a documented fallback").
     """
-    from llm.langchain_llm import build_langchain_llm
+    try:
+        from llm.langchain_llm import build_langchain_llm
+    except ImportError:
+        logger.warning(
+            "LangChain is not installed; falling back to the provider SDKs in "
+            "llm.cloud_llm. Install langchain-core and the integration package "
+            "for your provider to use the LangChain path."
+        )
+        return _build_llm_via_sdks(get_secret)
 
     return build_langchain_llm(get_secret=get_secret)
 
 
+def _build_llm_via_sdks(get_secret: Callable[[str], str]):
+    """Pre-LangChain provider selection, kept as the no-LangChain fallback."""
+    from llm.cloud_llm import PROVIDER_PRECEDENCE
+
+    for env_var, provider in PROVIDER_PRECEDENCE:
+        if not get_secret(env_var):
+            continue
+        try:
+            from llm.cloud_llm import CloudLLM
+
+            client = CloudLLM(provider, api_key=get_secret(env_var))
+        except Exception:
+            logger.exception("Failed to initialise cloud provider %s", provider)
+            continue
+        if client.is_available:
+            return client
+        # Key present but no usable client — say so instead of silently
+        # returning canned errors for every question.
+        logger.error(
+            "%s is configured but its client could not be created — either the "
+            "provider SDK is not installed (pip install %s) or client creation "
+            "failed; see the log above.", provider, provider,
+        )
+
+    try:
+        from llm.local_llm import get_llm as _get_local_llm
+
+        client = _get_local_llm()
+    except Exception:
+        logger.exception("Failed to initialise the local Ollama client")
+        return None
+    return client if client is not None and client.is_available else None
+
+
 # --- Search ------------------------------------------------------------------
 
 def run_search(profile: StudentProfile) -> List[Dict]:
@@ -252,7 +301,7 @@ _UNWANTED_SCRIPTS = re.compile(
     '一-鿿'   # CJK unified ideographs
     '가-힯'   # hangul syllables
     '豈-﫿'   # CJK compatibility ideographs
-    '-'   # private use area
+    '-'   # private use area
     '�'          # replacement character
     ']+'
 )
@@ -854,4 +903,4 @@ def free_chat_reply(message: str, visible: List[Dict], all_results: List[Dict],
     # checking it would reject its own shortlist names.
     if reply is not fallback and not reply_consistent_with_context(reply, context):
         return ChatTurn(fallback, visible)
-    return ChatTurn(reply, visible)
+    return ChatTurn(reply, visible)

tests/test_engine.py

@@ -587,8 +587,13 @@ def test_grounded_reply_passes_through():
 
 # --- build_llm provider selection -----------------------------------------
 
+# build_llm now delegates to llm.langchain_llm.build_langchain_llm, so these
+# patch LangChainLLM rather than cloud_llm.CloudLLM. The contract under test is
+# unchanged — precedence order and passing the shell-resolved secret through —
+# because langchain_llm.PROVIDER_PRECEDENCE is the same list.
+
 def test_build_llm_returns_none_without_keys(monkeypatch):
-    monkeypatch.setattr("llm.local_llm.get_llm", lambda: None)
+    monkeypatch.setattr("llm.langchain_llm.LangChainLLM", _unavailable_client)
     assert engine.build_llm(lambda name: "") is None
 
 
@@ -605,25 +610,36 @@ class _FakeCloud:
         return True
 
 
+def _unavailable_client(provider, **kwargs):
+    """Stand-in for a provider whose client cannot be built (no Ollama here).
+
+    Raises rather than returning an unavailable _FakeCloud: reaching into
+    _FakeCloud to flip its is_available rebinds the attribute on the class
+    every other test shares, so the ones that ran afterwards saw a client that
+    was never available.
+    """
+    raise RuntimeError("no client for " + provider)
+
+
 def test_build_llm_prefers_groq(monkeypatch):
     _FakeCloud.instances = []
-    monkeypatch.setattr("llm.cloud_llm.CloudLLM", _FakeCloud)
+    monkeypatch.setattr("llm.langchain_llm.LangChainLLM", _FakeCloud)
     client = engine.build_llm({"GROQ_API_KEY": "x", "OPENAI_API_KEY": "y"}.get)
     assert client.provider == "groq"
 
 
 def test_build_llm_falls_through_to_next_provider(monkeypatch):
     _FakeCloud.instances = []
-    monkeypatch.setattr("llm.cloud_llm.CloudLLM", _FakeCloud)
+    monkeypatch.setattr("llm.langchain_llm.LangChainLLM", _FakeCloud)
     client = engine.build_llm({"OPENAI_API_KEY": "y"}.get)
     assert client.provider == "openai"
 
 
 def test_build_llm_ranks_together_above_openai(monkeypatch):
-    """One precedence list now: cloud_llm.PROVIDER_PRECEDENCE. The engine used
-    to keep a second copy that omitted Together entirely."""
+    """One precedence list now: langchain_llm.PROVIDER_PRECEDENCE. The engine
+    used to keep a second copy that omitted Together entirely."""
     _FakeCloud.instances = []
-    monkeypatch.setattr("llm.cloud_llm.CloudLLM", _FakeCloud)
+    monkeypatch.setattr("llm.langchain_llm.LangChainLLM", _FakeCloud)
     client = engine.build_llm({"TOGETHER_API_KEY": "t", "OPENAI_API_KEY": "o"}.get)
     assert client.provider == "together"
 
@@ -632,6 +648,6 @@ def test_build_llm_passes_the_resolved_secret_to_the_client(monkeypatch):
     """Keys that live only in st.secrets must reach the SDK client: the
     client used to re-read os.getenv and silently found nothing."""
     _FakeCloud.instances = []
-    monkeypatch.setattr("llm.cloud_llm.CloudLLM", _FakeCloud)
+    monkeypatch.setattr("llm.langchain_llm.LangChainLLM", _FakeCloud)
     engine.build_llm({"GROQ_API_KEY": "resolved-by-the-shell"}.get)
     assert _FakeCloud.instances[0].kwargs["api_key"] == "resolved-by-the-shell"