American artificial intelligence start-up Thinking Machines Lab officially opened the Inkling 2 Small model weight download on November 12, using Apache 2.0 authorization, allowing developers to deploy and fine-tune themselves. This model was first released on November 5. It can only be used through the official Tinker platform initially, and the weight will be released after external security testing is completed. Inkling 2 Small has 594 billion total parameters (594B), each inference uses approximately 17 billion parameters (17B), supports up to 1 million token contexts, and mainly supports AI agents, program development, data analysis and enterprise automation applications.
Inkling 2 Small uses a mixed expert (MoE) architecture with 48 layers of Transformer. Each time it processes text, 8 experts will be selected from 384 to participate in the calculation, thereby reducing the amount of calculation required for inference. The model can understand text and images at the same time and output results in text. It also supports tool calls and different levels of inference settings. Developers can use it for chatbots, coding assistants or search-enhanced generation (RAG) systems as needed, but the new version does not provide audio input functionality.
Another feature of this model is that it is trained in the NVIDIA NVFP4 low-precision format, with the same accuracy from pre-training to post-training. Thinking Machines Lab offers both NVFP4 and MXFP8 weight versions, with NVFP4 being the officially recommended deployment choice because MXFP8 only improves by about 1 to 2 percentage points in multiple tests, but requires more memory. Depending on the model card, the NVFP4 version requires at least 500GB of GPU memory and can be executed with two NVIDIA B300 or four H200; MXFP8 requires at least 800GB. Although only 17 billion parameters are enabled at a time, the full model weights still need to be loaded, so it is not easily run on ordinary consumer-grade graphics cards.
In terms of performance, the official BrowseComp web search test achieved 90.2%, and the Finance Agent v2.1 financial agent test achieved 67.5%, both achieving the highest results among the models compared in the same field. The Full test of SWEBench Pro v2 reached 97.7%, but the more difficult Hard test was 86.3%, showing that there are still differences in performance under different test conditions. In another AA-Omniscience v1.1 evaluation, the non-hallucination rate of the model was 73.3%, which means that it is better able to avoid fabricating content out of thin air when answering incorrectly. It cannot be directly interpreted to mean that all answers have a correct rate of 73.3%.
The Inkling 2 Small didn't lead every test. In Artificial Analysis Intelligence Index v5.0, the model scored 30.6, slightly lower than some Chinese open weight models; the score of Terminal-Bench 4.0 was only 13.1%, which is still significantly different from other models that are good at terminal operation. Thinking Machines Lab also admitted that the new model still has room for improvement in complex tool selection, content organization, and general chat experience. In some situations, excessive refusal to answer questions may occur.
It is worth noting that Inkling 2 Small was developed by an American company and adopts the relatively loose Apache 2.0 license, providing Taiwanese companies and research institutions with another non-Chinese source model choice. However, just because it can be downloaded for free or deployed by yourself does not mean that it can be directly used for all government services. According to the Taiwan Executive Yuan's guidelines for the use of generative AI, public agencies still need to evaluate data security, personal information protection and use risks, and there are additional restrictions on the processing of confidential documents and undisclosed official information. If a closed ground deployment is adopted, the safety of the system environment must still be confirmed first and the internal management regulations of each agency must be followed.
