32 lines
5.8 KiB
Plaintext
32 lines
5.8 KiB
Plaintext
# Chat-parity gate corpus.
|
|
#
|
|
# Format: one document per blank-line-separated block; S<TAB> is the system prompt, U<TAB> a
|
|
# user turn, A<TAB> an assistant turn. tests/chat_corpus.c frames these with the MODEL'S OWN
|
|
# chat template and encodes them with its OWN BPE, so the gate can never be fed another
|
|
# tokeniser's ids the way the 20-probe gate was.
|
|
#
|
|
# Chosen for REPRESENTATIVENESS, not to make a number: documents 1-2 are the shape a serve
|
|
# actually receives (an S system prompt followed by one long technical question), documents
|
|
# 3-4 are multi-turn question-and-answer. Nothing here was selected because it showed the
|
|
# defect more loudly.
|
|
#
|
|
# The S role is load-bearing. The same instruction text emitted as a second USER turn instead
|
|
# of a system block does not reproduce the defect at all (ratio 0.925 against 1.221), so a
|
|
# corpus without a system prompt is not scoring what a serve sends.
|
|
|
|
S You are a helpful assistant. Answer directly and concisely. Do not think out loud.
|
|
U Explain how a CPU cache works.
|
|
|
|
S You are a careful technical assistant. Answer directly and concisely.
|
|
U I am debugging a performance problem in a C server and I want your help reasoning about it. The server accepts connections on a single listening socket, hands each connection to a worker thread from a fixed pool of sixteen threads, and each worker parses a request, looks up a key in a shared hash table protected by a reader-writer lock, serialises a response into a per-connection buffer, and writes it back. Under light load the median response time is about two hundred microseconds and the ninety-ninth percentile is about four hundred microseconds, which is fine. Under heavy load, at around forty thousand requests per second, the median stays near two hundred and fifty microseconds but the ninety-ninth percentile jumps to eleven milliseconds, and the machine is only about sixty percent busy across all cores, so it is not simply saturated. I have already checked that the network interface is not dropping packets and that the socket receive queues are not overflowing. The hash table is read-mostly: writes are under one percent of operations. Each response buffer is allocated from a global allocator on every request and freed at the end. The worker threads are not pinned to cores. Profiling with a sampling profiler shows about thirty percent of cycles inside the lock acquisition path, but the profile is flat and does not obviously point at one caller. Please walk me through the most likely causes of a tail latency of that shape, in order of how much I should suspect them, explain what evidence would confirm or rule out each one, and tell me which measurement I should take first and why. Be concrete about the mechanism in each case rather than giving general advice, and say explicitly which of these could produce a tail that is fifty times the median while leaving the median almost unchanged.
|
|
|
|
U What does a CPU cache do, and why are there several levels of it?
|
|
A A CPU cache is a small, fast memory that sits between the processor and main memory and keeps copies of recently used data and instructions. Main memory is roughly a hundred times slower than the core, so without a cache the processor would spend most of its time waiting. There are several levels because speed and capacity pull in opposite directions: the L1 cache is tiny, perhaps thirty-two kilobytes, and can answer in a few cycles; the L2 cache is larger and a little slower; the L3 cache is shared between cores and may be tens of megabytes, but it takes dozens of cycles to reach. Each level catches what the level above it missed, so the average access time stays close to the fastest level while the effective capacity stays close to the largest.
|
|
U How does the cache decide what to keep?
|
|
A Most caches use an approximation of least-recently-used replacement within a set. The address of a line selects a set, the line can go in any way of that set, and when the set is full the way that has gone longest without a reference is evicted. True least-recently-used tracking is expensive above four ways, so hardware usually keeps a cheaper approximation such as a tree of bits or a single reference bit per way. The important consequence for a programmer is that access pattern matters more than total working-set size: a loop that walks an array with a stride equal to the set spacing can thrash a cache that would happily hold ten times the data if it were touched in order.
|
|
|
|
U Explain why floating point addition is not associative.
|
|
A Floating point numbers carry a fixed number of significand bits, so almost every operation rounds. Rounding depends on the magnitudes of the two operands, and that makes the order of a sum matter. If you add a very large number to a very small one, the small one may be entirely lost to rounding; add the small numbers together first and their sum may be large enough to survive. A classic example is adding one to a large value and then subtracting the large value again: you get zero rather than one, because the intermediate result could not represent the difference. This is why numerical libraries use compensated summation, and why a parallel reduction that changes the order of a sum can produce a slightly different answer from the sequential version even when both are correct.
|
|
U Does that mean parallel reductions are unreliable?
|
|
A Not unreliable, but not bit-reproducible. Each ordering is a valid answer within the error bounds of the operation, and the spread between orderings is usually far smaller than the error already present in the inputs. Where it matters is testing: a gate written as a bit-exact comparison will fail the moment the reduction tree changes, even though nothing is wrong. The usual fix is to compare against a tolerance derived from the condition number of the sum, or to fix the reduction order when reproducibility is worth more than speed.
|