Match changes in model depth to the cost and outcome you need
A language assistant may need a faster answer; a tabular predictor may need a smaller model. Adding a pass through shared weights increases computation without adding stored weights, while removing blocks changes the model itself. The same word, depth, hides different costs.
These studies suggest choosing a change at the point where the cost arises, then judging it by the outcome actually tested.
Their evidence covers training loss, downstream language tasks, word prediction and tabular classification under different conditions; it does not identify one best depth for every model.
What you will understand
How does spending computation on training loops differ from adding loops when a trained model answers a request?
Why should a candidate for block removal be judged with its intended replacement or repair in place?
Which measured outcome and study limits matter when choosing a depth change for a particular system?
Decide where the computation is spent
Imagine serving a language assistant that sometimes has time for a longer answer, alongside a tabular classifier running on a local clinical server. The assistant's operator might spend more computation on a difficult request.
The classifier's operator might need to reduce the model's stored size or the work required for each prediction. Both face a depth decision, but changing depth can mean repeating existing blocks or removing blocks from the stored model.
A repeated block uses the same learned weights on successive passes. More passes therefore add work without requiring a separate set of weights for each pass.
The timing of those passes matters: loops during training consume training computation, whereas loops when answering a request consume inference computation and time. A model that can be served at different loop counts raises a different question from a training study that divides a fixed budget among passes and training examples.
Removing blocks poses another choice. Skipping a section changes the signals received by later parts of the model, so a method may fit a small transformation to stand in for the skipped computation.
That can reduce computation and the weights stored for blocks, depending on the replacement's size. The shared question is therefore practical: where should depth change to meet a cost target, and what evidence shows that the outcome of interest survives?
Where the lesson goes next
Use extra inference loops for a measured gain
Choose removals after accounting for repair
Test replacement against the full predictor
Match the choice to its tested outcome