Today I'm posting a comparative hands-on test of open vs. closed-source models.
Sonnet The series was last updated in February; this time, with the lifting of restrictions on Fable 5, another version has been updated. From the data released by Claude , overall it is stronger than Oups 4.6 and 4.7, and a bit weaker than Oups 4.8, but in actual operation, Sonnet costs even more than Opus.
Sonnet 5A new tokenizer is used, so the same text is now split into more tokens, and the calculated total cost is higher than before.
I selected four open-source models:Kimi K2.7 Code, GLM 5.2, MiniMax M3, DeepSeek V4 Pro to run a comparative hands-on test against Sonnet 5, to see just how good Sonnet 5 really is.
01. Hands-On Comparison
case 1 Bug Fix
Prompt:
Below is a piece of Python code used to convert amount strings into cents. In production, it was found that "1,234.50", "$0.99", " -12.30 ", and "12" produce errors or inconsistent results. Please fix the function and add pytest tests.
def parse_money_to_cents(value: str) -> int:
value = value.replace(“$”, “”)
dollars, cents = value.split(“.”)
return int(dollars) * 100 + int(cents)
Sonnet 5：
Sonnet 5's calculation method is very stable, using Decimal to handle amounts and avoid floating-point errors. The problem is that input cleaning is too permissive: 1,2,3 is treated as 123, 1$2 is treated as 12, and $$12 is also treated as 12, so it is not defensive enough against dirty data.
Kimi K2.7 Code：
Kimi K2.7 Code has format checking and also rejects amounts with more than two decimal places, such as 12.345. But the approach of first deleting $ and commas and then checking the format causes 1,2,3, 1$2, and $$12 to be cleaned into valid numbers. Moreover, +$12.30, a plus sign plus currency symbol, is rejected.
GLM 5.2：
GLM 5.2 fixed the core problems in the prompt; for example, 1,234.50, $0.99, -12.30, and 12 can all be handled. However, it does not perform a complete format check first. For example, –12.30 is calculated as a positive number, 12.-3 produces a result, and 1,2,3 is also accepted.
MiniMax M3：
MiniMax M3 first checks whether the input matches a normal amount format before proceeding to calculation, so messy commas like 1,2,3 are rejected and will not be mistakenly calculated as 12300 cents. But the rules are slightly conservative: common user shorthand such as .99 and $.99 is rejected, and symbol positions such as $-12.30 are not compatible.
DeepSeek V4 Pro：
DeepSeek V4 Pro's main flow can run through, but its edge cases are the weakest. For example, an empty string, whitespace, or a lone $ all return 0, which is very dangerous in amount scenarios; $-12.30 is calculated as -1170, not -1230; –12.30 is also incorrectly accepted, lacking a layer of "first determine whether the input is valid."
Ranking for this case: MiniMax M3 > Sonnet 5 > Kimi K2.7 Code > GLM 5.2 > DeepSeek V4 Pro
case 2 Boundary Condition Reasoning
Prompt: Please implement a TypeScript function groupByWindow(events, windowMs). events is an array sorted in ascending order by time, and each item is {id, ts}. The function should group adjacent events: as long as the difference between the ts of the current event and the ts of the first event in the current group is less than or equal to windowMs, put it in the same group; otherwise, start a new group. Please provide the implementation and 6 boundary tests.
Sonnet 5：
Sonnet 5's approach is very clear, and it especially points out the "chain drift" problem. It also covers an empty array, a single element, being equal to the boundary, exceeding the boundary, and windowMs=0. But it lacks a bit of input protection; for example, how to handle a negative window is not explained.
Kimi K2.7 Code：
Kimi K2.7 Code's implementation is correct, and the project is very complete, with package.json, tsconfig, and Vitest all configured. But the types are relatively ordinary, and the return value loses the types of extra fields on events.
GLM 5.2：
GLM 5.2's grouping mechanism is very clear: each time it compares the current event with the 'first item in this group', and it also checks windowMs; negative numbers and infinite values will directly throw an error, which is quite suitable for real functions. Types are also well preserved: whatever event type is passed in, the corresponding type is returned.
MiniMax M3：
MiniMax M3's core logic is correct, and it also has a large number of tests, covering windowMs=0, exactly equal to the window, just exceeding the window, and the group-first anchor, and it also checks whether each event's ts is a valid number. However, the id type is written as string, so numeric id scenarios cannot pass at the type level; and windowMs does not block negative numbers.
DeepSeek V4 Pro：
DeepSeek V4 Pro's function itself has the correct main logic, using the first item of the last group as the baseline. But the test file contains garbled text, the assertion for the empty array case was swallowed by a comment, and the beginning of the fifth assertion was also swallowed by a comment, so the following statements look like they cannot run; in the sixth one, the line with id:5 was also commented out, so the input and expectation do not match.
Ranking for this case: GLM 5.2 > Sonnet 5 > Kimi K2.7 Code > MiniMax M3 > DeepSeek V4 Pro
case 3 Code Review
Prompt:
Please review the following Node.js code and identify issues that could cause production incidents, security problems, or data corruption. Sort by severity, and for each issue give the cause and a minimal fix.
app.post(‘/transfer’, async (req, res) => {
const { from, to, amount } = req.body
const user = await db.user.findFirst({ where: { id: from } })
if (user.balance < amount) return res.send(‘no money’)
await db.user.update({ where: { id: from }, data: { balance: user.balance – amount } })
const target = await db.user.findFirst({ where: { id: to } })
await db.user.update({ where: { id: to }, data: { balance: target.balance + amount } })
res.send(‘ok’)
})
Sonnet 5：
Sonnet 5 grasped the core mechanism issues in the transfer code: money-related operations must simultaneously satisfy 'debit succeeds, credit succeeds, rollback on failure, and no over-debit under concurrency'. Its proposed solution wraps both updates in a transaction and combines 'whether the balance is sufficient' and 'debit' into one step, which can block concurrent double-spending. Authentication, negative amounts, nonexistent accounts, status codes, and duplicate submissions are also all covered.
Kimi K2.7 Code：
Kimi K2.7 Code focuses on the right key points: authentication source, input validation, atomic debit, and transactional credit. It clearly states that from should come from the logged-in user and must not be trusted from the request body, which is crucial. A minor issue is that the handling of amount units is a bit jumpy: it first says the API amount may be in yuan, then converts it to cents, but the examples assume the database balance is already in cents.
GLM 5.2：
GLM 5.2 shows that it is thinking about a real production system: concurrency, insufficient balance, nonexistent target account, authentication, amount precision, idempotency, rate limiting, and timeouts are all mentioned. The mechanism explanation is also detailed, but the remediation plan is a bit scattered, and ranking authentication fifth is also too low; anyone being able to specify from and transfer away someone else's money should be at the very top.
MiniMax M3：
MiniMax M3 mentions idempotency, audit logs, rate limiting, and error bodies, like a pre-launch security checklist. The problem is that several statements are not solid enough: in the Prisma example, it puts balance: { gte: amount } into update.where; and the part about req.body not being parsed is also judged incorrectly, since destructuring undefined will directly throw an error.
DeepSeek V4 Pro：
DeepSeek V4 Pro caught the main issues: no transaction, no authentication, negative amounts, nonexistent target account, floating-point amounts, and idempotency. But for the first fix: DeepSeek V4 Pro still reads the balance first, then checks it, then debits. Under concurrency, errors can still occur. At minimum, it should use a single-step update of 'debit only if the balance is sufficient'.
Ranking for this case: Sonnet 5 > Kimi K2.7 Code > GLM 5.2 > MiniMax M3 > DeepSeek V4 Pro
case 4 Code Refactoring
Prompt:
Please refactor the following function into a more maintainable version, with unchanged behavior, and write the test cases you would add. Do not introduce third-party libraries.
function price(order) {
let total = 0
for (const item of order.items) {
if (item.type === ‘book’) total += item.price * item.qty * 0.9
else if (item.type === ‘food’) total += item.price * item.qty
else if (item.type === ‘luxury’) total += item.price * item.qty * 1.2
else total += item.price * item.qty
}
if (order.country === ‘US’) total *= 1.07
if (order.country === ‘DE’) total *= 1.19
if (order.vip) total *= 0.95
return Math.round(total * 100) / 100
}
Sonnet 5：
Sonnet 5 is the most on-topic: it splits functions, uses lookup tables, and preserves the original calculation order and Math.round(total * 100) / 100. It also explains why the loop accumulation order, post-tax VIP, and default values for unknown types are all unchanged.
Kimi K2.7 Code：
Kimi K2.7 Code passed all 7 tests, and the code preserved the original behavior, so it counts as a qualified refactor. But the test coverage is on the conventional side; it did not separately stress the default food branch and the small but common behavior of no country.
GLM 5.2：
All 11 GLM 5.2 tests passed, the code is very clean, and the types were filled in appropriately. ITEM_MULTIPLIERS / TAX_RATES make it easier to add new categories and countries, and overall no requirements were added haphazardly.
MiniMax M3：
MiniMax M3 has a strong software engineering flavor, and all 22 tests passed, but this round's task was a "refactor," and MiniMax proactively changed behavior: it added input validation, restricted the vip type, rejected negative and decimal quantities, and also switched rounding from Math.round to toFixed.
DeepSeek V4 Pro：
DeepSeek V4 Pro ran the tests locally with vite-node, and actually all 13 passed. The implementation direction is fine: table-driven, single-item calculation, tax, VIP, and rounding were all preserved. It also separately tested no country, combined scenarios, multiple types, and two rounding examples.
Ranking for this case: Sonnet 5 > GLM 5.2 > DeepSeek V4 Pro > Kimi K2.7 Code > MiniMax M3
case 5 Chinese writing
Prompt:
Please rewrite the following product introduction into a natural, persuasive Chinese official website copy. Requirements: no AI tone, no vague adjectives, and no words such as "empower, closed loop, reshape, ultimate." Keep the key information and keep it within 180 characters.
[Modou can organize scattered materials, extract article structure, rewrite in different tones, check for AI flavor and sensitive expressions, and can also save the author's commonly used words and fixed expressions.]
Sonnet 5：
Sonnet 5 starts from pain points such as materials being scattered in memos, chat records, and audio transcriptions, and then naturally brings in organizing, rewriting, checking for AI flavor, and saving personal expressions, much like real product copy.
Kimi K2.7 Code：
Kimi K2.7 Code meets the requirements in length, object, function, and scenario, and did not add any arbitrary settings.
GLM 5.2：
GLM 5.2's: "a few fewer rounds of rework," this sentence is good. But "40-minute interview, twenty minutes, ready to publish directly" are features it added on its own.
MiniMax M3：
MiniMax M3 exceeds 180 characters, and also added settings that were not given, such as desktop version and sensitive word list. It also reads like a product page detail, not a product introduction.
DeepSeek V4 Pro：
DeepSeek V4 Pro reads more like a blogger recommendation; among them: "throw it all in at once, of course it's different" is too colloquial, a bit far from the attribute of an official website introduction.
Ranking for this case: Sonnet 5 > Kimi K2.7 Code > GLM 5.2 > MiniMax M3 > DeepSeek V4 Pro
case 6 data analysis
Prompt:
Below is 8 weeks of data for a certain product. Please judge whether the growth is healthy and point out the problem that should be prioritized for solving.
week, visitors, signup_rate, activation_rate, paid_rate, churn_rate
1, 10000, 8.0%, 42%, 6.0%, 3.0%
2, 12000, 7.5%, 39%, 5.8%, 3.4%
3, 15000, 6.8%, 35%, 5.2%, 4.1%
4, 18000, 6.1%, 31%, 4.7%, 4.8%
5, 22000, 5.4%, 28%, 4.2%, 5.6%
6, 26000, 4.9%, 24%, 3.8%, 6.3%
7, 30000, 4.4%, 21%, 3.4%, 7.1%
8, 35000, 3.9%, 18%, 3.0%, 8.0%
Sonnet 5：
Sonnet 5 caught the signal that all conversion rates are sliding down smoothly together, and prioritized suspecting low-quality traffic. However, the paid user count algorithm has a definition issue: it says it calculates by sequential funnel, but in the table the paid user count was not multiplied by the activation rate.
Kimi K2.7 Code：
Kimi K2.7 Code briefly and clearly explains the assumption for paid_rate. But the judgment of the cause of traffic quality is placed too far back.
GLM 5.2：
GLM 5.2's arithmetic is basically correct, and the analysis is also complete, but the conclusion is biased toward "activation is the only bottleneck," making it easy to miss the larger clue of the channel change from week 2 to week 3.
MiniMax M3：
MiniMax M3 can see traffic quality and activation handoff, but the key calculation is wrong.
DeepSeek V4 Pro：
DeepSeek V4 Pro calculated that the paid user count in week 8 dropped from about 20 to about 7, and judged: visits rose, while the absolute number of activations and paid users declined, which is very accurate. The mechanism is also explained clearly: the problem appears on the surface to be the activation rate, but the root is most likely traffic quality, and the handling action also lands on splitting by channel.
Case ranking: DeepSeek V4 Pro > Sonnet 5 > Kimi K2.7 Code > GLM 5.2 > MiniMax M3
Case 7 Web page generation
Prompt:
Please create a single-page product official website. The brand is called VelaRun, and the new product is a pair of urban training running shoes, VelaRun Tempo.
Requirements:
1. The first screen must let users see the shoes at a glance; do not hide the product in a card or a small image.
2. The page includes:
– Top navigation
– Hero section: large product image, product name, price, color selection, size selection, add to cart
– Selling points section: cushioning, breathability, stability, weight
– Image story section: urban running scenes, do not use abstract illustrations
– Specifications section
– User reviews section
– Fixed purchase bar on mobile
3. Interactions:
– Switching colors updates the main image and selected state
– Clicking buy without selecting a size should show a prompt
– After adding to cart, show a small toast
4. Style: clean, sporty, must not look like a template site. Avoid large blue-purple gradients and stacked rounded cards.
5. Responsiveness must be complete; images, buttons, and text must not overlap on mobile.
6. Implement with React + TypeScript, write the CSS yourself.
7. Provide complete runnable code.
Sonnet 5：
Sonnet 5's shopping mechanism is very complete: colors, sizes, missing-size prompts, sold-out sizes, cart quantity, toast, and the mobile purchase bar are all present, and the page structure is complete.
Kimi K2.7 Code：
Kimi K2.7 Code's layout is also basically complete, but it feels very template-like. However, the cart has no quantity feedback, and the interactions only cover color, size, and prompts.
GLM 5.2：
GLM 5.2's visuals are clean, and content such as specs, selling points, and the city testing story feels more like a real product page. But the interactions are very shallow, the shopping bag number is static, and the page feels more like a showcase page.
MiniMax M3：
MiniMax M3 is in a dark style, the components are clearly split, the page has plenty of information, and the buy and wishlist buttons also have an e-commerce feel. But I do not understand this green design in the middle.
DeepSeek V4 Pro：
Aside from DeepSeek V4 Pro drawing the shoe with SVG, and it looks a bit like a simplified model, with texture not as good as a real product image, everything else is very good.
Case ranking: Sonnet 5 > DeepSeek V4 Pro > MiniMax M3 > GLM 5.2 > Kimi K2.7 Code
Case 8 3D animation
Prompt:
Please create a single-page 3D product official website. The product is called EchoStone, and it is a desktop smart speaker.
Technical requirements:
1. Use React + TypeScript + Three.js.
2. The first screen must be a full-screen 3D scene; do not put the 3D in a small card.
3. The 3D scene must include a rotatable smart speaker model. You can model it by combining basic Three.js geometries; do not rely on external 3D files.
4. The speaker needs to include:
– Cylindrical main body
– Top touch ring
– Speaker mesh texture or hole array
– A glowing status light
5. Interactions:
– Drag with the mouse to rotate the product
– Scroll wheel zoom with limits
– Click color buttons to switch the body color
– After clicking "Play Demo", the status light and soundwave animation start moving
6. Page content:
– Top navigation
– Hero copy overlaid on the 3D scene
– Color selection
– Three selling points
– Technical specifications section
7. Style: premium consumer electronics, restrained, clean, no blue-purple gradients, no glowing orb background.
8. Responsive: on desktop and mobile, the 3D composition must not be blank, severely cropped, or cover the main buttons.
9. Provide complete runnable code.
Sonnet 5：
With Sonnet 5, the 3D product hero section came out right away—cylindrical body, top touch ring, mesh holes, light strip, and sound wave rings are all there, and drag rotation and scroll zoom are complete. Mechanically, it has resize, resource cleanup, and instanced mesh holes, like a product demo that can continue to be maintained.
Kimi K2.7 Code：
Kimi K2.7 Code is fully featured: OrbitControls, color switching, and sound wave animation are all there, and it runs overall. The problem is that the structure is somewhat concentrated, and the visual polish is a bit rough.
GLM 5.2：
GLM 5.2's top scale marks and mesh holes are missing. The direction the sound waves come out is a bit odd, and the model is too large, pressing onto the title and controls.
MiniMax M3：
MiniMax M3's model details and animation mechanics are strong: the base, light strip, and floating motion are all done, but the sound waves were not produced.
DeepSeek V4 Pro：
For DeepSeek V4 Pro, for some reason, the model has an extra ring, and this is a big deduction. Everything else is pretty good; it has what it should have.
Ranking for this case: Sonnet 5 > MiniMax M3 > Kimi K2.7 Code > GLM 5.2 > DeepSeek V4 Pro
02. Evaluation of each model
Sonnet 5 still looks relatively stable. Among the 8 cases, Sonnet 5 ranks first on comprehensive tasks such as code review, code refactoring, Chinese writing, web page generation, and 3D animation; in the few items where it did not take first place, it basically ranked second.
The four open-source models each have their strengths:
- GLM 5.2 is the best at boundary conditions and type protection.
- Kimi K2.7 Code executes stably, and its tests and explanations are relatively complete.
- MiniMax M3 easily gives solutions with a strong engineering feel, and both amount parsing and 3D model details are good.
- DeepSeek V4 Pro performs best in data analysis.
03. Digging deeper
Open-source models can already take on a lot of real work. GLM 5.2's boundary reasoning is very detailed, Kimi K2.7 Code delivers completely, MiniMax M3 has surprises in visuals and engineering, and DeepSeek V4 Pro is very sharp when making data judgments.
Looking at a single case, Sonnet 5 is often caught up to, or even surpassed.
But looking at the 8 cases together, Sonnet 5's advantage remains. In tasks such as code review, refactoring, Chinese writing, web page generation, and 3D animation, Sonnet 5 makes fewer low-level mistakes and is better able to accommodate subtle requirements.
This kind of stability directly affects rework time, launch risk, and manual review costs.
So when the price difference is large, the way to choose actually becomes clearer. For low-risk, high-frequency tasks that can be verified in batches, you can first hand them to open-source models.
High-risk tasks are more suitable for Sonnet 5, such as complex refactoring, pre-launch code review, pages that need to be delivered directly to clients, and content with high requirements for copy tone.
The best way to save money is to run the models separately: open-source models are responsible for running more, trying more, and producing more, while Sonnet 5 is responsible for key judgments and final closure.
Original link:The Sonnet series has been updated to 5, and I selected four domestic open-source models to PK with it