In a recent social media post, Z.AI co-founder and Chief Scientist Tang Jie shared his detailed perspective on the industry's Scaling Law philosophy, offering a candid look at why the company chose to keep its latest model's parameter count unchanged.

Tang emphasized that evaluating model scale requires more than just looking at parameter numbers. He argued that training data volume, computational resource allocation, and the end-user's operating environment are equally critical factors that must be considered together.

Reflecting on the trajectory of AI development, Tang noted that the industry's pursuit of trillion-parameter models, influenced by early research from 2020 and models like GPT-3, Gopher, and MT-NLG, was a collective detour that was later reversed. The 2022 Chinchilla study provided a more balanced perspective on the relationship between data and parameters, but Tang believes this was not the final answer.

As models are now called upon billions of times daily, Tang pointed out that inference costs have become a dominant factor over a model's lifecycle. This shift makes smaller models with extended training periods more optimal. Furthermore, the rise of Mixture-of-Experts (MoE) architectures has made it necessary to distinguish between total parameters and the parameters actually activated during each inference, as they influence knowledge storage and reasoning depth, respectively.

For complex tasks like vulnerability discovery, Tang stressed that merely memorizing more information is insufficient; models need the capacity for sustained, multi-step reasoning. He used this principle to explain the strategic decision behind GLM-5.3.

According to Tang, Z.AI did not alter the base model, architecture, total parameters, or activated parameters of GLM-5.2 for the new GLM-5.3. Instead, the focus for a month was on expanding long-horizon task environments and reinforcement learning. Tang described the resulting performance gains as significant, not merely incremental.

Using a metaphor of adjusting multiple knobs, Tang explained that this time Z.AI prioritized post-training due to its remaining potential, while clarifying that other directions like parameter scale, pre-training data, and per-forward-computation depth are still important. He assured that future efforts would revisit these areas, with the next expansion possibly targeting mid-training or pre-training phases.

Tang's post sparked a lively discussion among users. One inquired about when post-training might see its own Chinchilla-like Scaling Law research and whether the reinforcement learning curve for GLM has plateaued.

Another user pointed to models like Qwen 3.8 27B and GLM-5.3 as evidence that mid-sized models still have room for improvement, suggesting that visual capabilities and operational speed could be the next upgrade targets for the GLM series.

A third commenter highlighted the importance of questioning itself, noting that the ability to formulate good questions is rarer than providing answers, and that deeper structural issues deserve more attention than surface-level responses.

Here is the full translation of Tang Jie's post, edited for clarity:

Reflections on Scaling Law

Model scale is still growing, but what is expanding should not be limited to parameter count. Every time a model is released, the same question arises: how many parameters does it have? This question cannot be answered in isolation. Parameter count only makes sense when combined with three other questions: how much training data is available, where the computational resources are being allocated, and who will run the model under what conditions.

The field has learned this through costly experience. Research by Kaplan et al. in 2020 suggested that parameter count should grow faster than data, with scaling exponents of 0.73 and 0.27 respectively. The industry followed this direction, including models like GPT-3, Gopher, and MT-NLG. However, in 2022, Hoffmann et al. re-examined around 400 models and found the compute-optimal configuration was closer to 20 tokens per parameter. With sufficient computational resources, parameter count and data volume should grow at similar rates, not diverge increasingly.

The errors in early fitting amplify with each order of magnitude increase in computation. Consequently, the largest models of that generation were also the ones with the most inefficient resource allocation. In hindsight, the race toward trillion-parameter models was a collective detour from which the industry subsequently retreated.

Yet Chinchilla is not the endpoint. It optimizes compute for the training phase, assuming a model is trained once and then evaluated. Now that a model may be called billions of times daily, inference has become the dominant cost over its entire lifecycle. If inference costs are included in the optimization target, the optimal solution shifts toward smaller models with longer training periods. This is an intentional form of "overtraining." Llama-2-7B and Gemma-2-9B follow a similar approach, with approximately 290 and 889 training tokens per parameter, respectively.

The emergence of sparse models has again changed the optimization goals. In MoE models, two metrics must be distinguished: total parameters generally determine how much knowledge the model can hold, including factual and long-tail knowledge; activated parameters and effective computation depth roughly determine how deep the model can reason and how many steps of causal inference it can sustain without collapsing. Therefore, the 20 tokens per parameter ratio for dense models cannot be directly applied to MoE models. This ratio is not fixed either.

Research by Roberts et al. in 2025 found that the optimal token-parameter ratio depends on task type: memory tasks benefit more from increased parameters, while reasoning tasks benefit more from increased training data. Subsequent studies on MoE have also found that, with a constant token-parameter ratio, increasing total parameters can actually weaken reasoning ability; increasing the number of activated experts per inference, however, can more consistently improve reasoning performance.

This aligns with the capabilities we are building. Finding a security vulnerability is not an information retrieval problem. It does not depend on how many CVE records the model has memorized, but on whether it can complete a chain of reasoning containing 20 steps while maintaining coherence until reaching the end. This capability does not reside in the total parameter count itself.

Now, about GLM-5.3. Total parameters remain important until a certain threshold is reached; the model first needs sufficient capacity to contain its understanding of the world. But beyond that threshold, additional capabilities must come from other directions, including improving the effective depth of each forward computation, especially expanding post-training. GLM-5.3 is a controlled experiment based on this judgment.

GLM-5.3 uses the same base model, architecture, total parameters, and activated parameters as GLM-5.2. We spent a month expanding the long-horizon task environment and reinforcement learning training, and the resulting improvement was not a minor adjustment.

Scaling does not have just one knob. This time, we chose to turn up the post-training knob because it still has the most room for improvement, not because other directions are complete. Base model scale, pre-training data, and the computational investment per forward pass are all still expandable, and we will return to these areas. This experiment taught us that these knobs do not need to be turned simultaneously; the indicator most worth adjusting next is usually not the one that was most worthwhile last time.

We have not stopped expanding. Next time, it may be mid-training, pre-training, or even more.