AI Research
Paper · 26 October 2026
Is AI coding affecting SaaS yet?
Our research into what the evidence actually shows: selective substitution and more competition, not an industry-wide revenue collapse. Read the paper and download the PDF.
The AI model landscape
SoftHouse Advisory's relative benchmarking of the core frontier models. Updated 2026-11-17.
Core model benchmarking
Snapshot 2026-11-17 · 0-10 relative, higher is betterThird base-pull (prior pulls: 2026-06-24, 2026-10-26). Scores are comparative across the five tracks in THIS snapshot, not absolute capability percentages. Several dimension scores this cycle are marked estimated in score_rationale where sources disagree or no benchmark was located; those are directional, not calibrated. Microsoft 365 Copilot is scored as a product/distribution/governance layer over partner frontier models, not as a standalone model, consistent with prior snapshots.
| Model | Reasoning | Code | Agentic | Tool Use | Cost / Value | Context Window | Governance Readiness | Composite |
|---|---|---|---|---|---|---|---|---|
Claude Opus 5.5 Anthropic · claude-opus-5-5 | 7 | 9 | 7.5 | 7 | 6 | 9 | 6.5 | 7.4 |
Microsoft 365 Copilot Microsoft · Copilot product layer (multi-model router: Grok 4.7, GPT-6 Astra, Claude Sonnet/Opus 5.5, Fable 5.1, among others) | 7 | 7.5 | 7 | 8 | 4.5 | 7 | 8 | 7.1 |
Gemini 4 Argon Google · gemini-4-argon | 6.5 | 7 | 6.5 | 6 | 6 | 9.5 | 5 | 6.5 |
GPT-6 Astra OpenAI · gpt-6-astra | 6.5 | 7.5 | 6.5 | 6.5 | 3.5 | 9.5 | 5 | 6.4 |
Grok 4.7 xAI · grok-4.7 | 6 | 7 | 7 | 6.5 | 7 | 4 | 4 | 6.1 |
Scores are 0–10 relative across the models in this snapshot, not absolute capability. Cost is normalised so higher = better value. Weighted mean of the seven dimension scores using dimension.weight (sum of weights = 7.0), rounded to 1 decimal place..
Claude Opus 5.5
confidence: medium- Leads SWE-bench Pro at 89.9%, the single highest sourced coding figure across this snapshot (third-party leaderboard, benchmark attributed to Anthropic's own system card).
- SWE-bench Multilingual 93.9%, also leading this snapshot.
- On an independent FrontierSWE v2 run, scores 62.3% mean@5, ahead of Gemini 4 Argon's 55.0% though behind GPT-6 Astra's 65.5%.
- Confirmed pricing of 4 USD per million input tokens and 20 USD per million output tokens (third-party pricing trackers), the second cheapest of the four standalone models in this snapshot.
- Anthropic publishes a system card and a versioned Responsible Scaling Policy for each release, a governance-transparency practice not matched by all peers this cycle.
- Anthropic did not report SWE-bench Verified, GPQA Diamond, MMMLU, tau-bench or ARC-AGI for Opus 5.5 in its announcement materials located this pass.
- Sources disagree on its Terminal-Bench 4.0 figure (66.4% vs 65.15%) depending on harness, so that number is not usable as a single figure.
- Pricing and context-window figures come from third-party trackers rather than Anthropic's own pricing page directly; one tracker lists an older, higher rate that appears to describe the prior Opus 5 generation rather than 5.5.
Microsoft 365 Copilot
confidence: low- Usage-based billing access to additional models including GPT-6 Astra and Claude Fable 5.1 is rolling out to Copilot Chat and Copilot in Word, Excel and PowerPoint starting November 2026 (Microsoft Message Center, marked in development as of this pass).
- Copilot in SharePoint and OneDrive reached general availability in October 2026.
- Deep enterprise data grounding via Microsoft Graph (tenant files, email, meetings, chat), embedded directly in Word, Excel, PowerPoint, Outlook and Teams.
- Every action inherits the user's existing security context plus Microsoft's enterprise admin and compliance tooling.
- Not a standalone frontier model: headline capability is entirely a function of which underlying model is selected, so its scores here reflect an inherited blend, not a native model's own benchmark result.
- Since around July 2026, the Copilot Chat model picker has in some tenants replaced named model labels (e.g. GPT, Claude) with response-style options (Auto, Quick, Think Deeper); visibility of the underlying model name now varies by tenant, a reduction in transparency about which model is actually answering compared to the prior snapshot.
- No independent benchmark exists for Copilot as a product, only for the models it routes to.
Gemini 4 Argon
confidence: low- 1,048,576-token context window shared across input, thinking and output (vendor docs).
- DeepSWE v1.1 77.9%, the highest sourced DeepSWE figure across this snapshot, ahead of GPT-6 Astra's 74.1% and Grok 4.7's 71.0%.
- LABBench 2 88.8%, a science-reasoning benchmark, the highest figure located for this track this pass.
- One independent index places it first on the Vals Index at 68.90%, narrowly ahead of a Claude Sonnet model.
- On an independent FrontierSWE v2 run, scores 55.0% mean@5, the lowest of the three models independently cross-checked (behind GPT-6 Astra's 65.5% and Claude Opus 5.5's 62.3%).
- Terminal-Bench 4.0 57.4%, behind Claude Opus 5.5's 66.4%.
- Sources disagree on availability: one source describes a 13 October 2026 general-availability rollout to subscribers, while others describe limited or pre-release API access with no announced general public release date; this is unresolved this pass.
- Scheduled price increase: introductory pricing roughly doubles from 1 December 2026 (2 USD to 4 USD per 1M input tokens, 10 USD to 20 USD per 1M output).
GPT-6 Astra
confidence: medium- 1.05M-token context window with up to 128K max output tokens, confirmed on OpenAI's own developer docs page this pass.
- DeepSWE v1.1 74.1%, previously the highest sourced DeepSWE figure, now second to Gemini 4 Argon's 77.9%.
- On an independent FrontierSWE v2 run, leads the three models cross-checked at 65.5% mean@5.
- Terminal-Bench Science 0.1 68.1%, the highest science-focused agentic figure located this pass.
- OpenAI has not published a SWE-bench Pro or SWE-bench Verified figure for this model as of this pass, despite that being a standard coding benchmark competitors report.
- Pricing sources still conflict (10 USD/50 USD per 1M vs a 5 USD/25 USD outlier last updated in September, likely stale); the figure here uses the better-corroborated, more recently dated rate.
- Remains the most expensive of the four standalone models in this snapshot on the better-corroborated pricing figure.
Grok 4.7
confidence: medium- Cheapest standalone pricing in this snapshot at 2 USD / 6 USD per 1M tokens (in/out), unchanged since launch.
- DeepSWE v1.1 71.0% (high-effort mode), up from 65.2% on its own prior generation (vendor-reported launch comparison).
- Terminal-Bench 4.0 38.0%, up from 20.3%, the largest generational jump found in this snapshot (vendor-reported).
- CursorBench 4.0 46.3%, up from 40.4% on its own prior generation.
- Knowledge cutoff of May 2026, several months stale against peers in this snapshot.
- 500K-token context window, the smallest of the four standalone tracks.
- Long-context requests at or above 200K tokens double the input/output price for the entire request, not just the overage.
- Independent coverage places it behind Claude and GPT-6-generation models on CursorBench and a separate AA-Briefcase benchmark, despite the vendor's own self-versus-prior-generation framing.
Landscape research
The EU's Digital Omnibus on AI is now formally in force as Regulation (EU) 2026/1744, published in the Official Journal on 24 July 2026 and in force from 27 July 2026. Beyond pushing back the high-risk deadlines (standalone high-risk systems now from 2 December 2027, products already covered by EU product-safety law from 2 August 2028), the finalized text adds detail that was not settled at the last review: two new prohibited AI practices covering systems that generate certain categories of illegal content; the existing small-and-medium-enterprise exemptions from some administrative requirements now extend to small mid-cap enterprises; the AI literacy duty is now framed as an obligation of effort rather than a guaranteed outcome; bias-detection data processing rules extend to deployers as well as providers, with a reinstated strict-necessity threshold for handling special-category data; and the EU AI Office gets exclusive supervisory authority over AI systems built on a general-purpose AI model from the same provider or corporate group. In the United States, there is still no federal AI statute, and the FTC's July 2026 policy statement questioning whether state AI laws can compel ideologically adjusted model outputs remains a proposed, non-binding position with the underlying federal-preemption question unresolved. California's SB 53 continues to require large frontier developers to publish safety frameworks and report critical incidents within 15 days, or 24 hours if risk is imminent. Frontier labs' voluntary safety frameworks continue to be updated on their own schedules and remain voluntary with no external enforcement. For mid-size firms, the practical change since the last review is that the EU's compliance calendar is now a settled regulation rather than a provisional agreement, so a plan built around the EU's new 2027 to 2028 dates can be treated as reasonably stable, while the US federal versus state question remains genuinely open.
Updated 2026-11-17 · 7 sources
Adoption data published since the last review deepens rather than overturns the earlier picture. JetBrains' research, covering developers surveyed May to July 2026, found 90% of professional developers use AI coding agents at work at least weekly, and 68% daily, an unusually high and consistent figure for a fast-moving category. A separate compilation puts enterprise-wide figures lower and more uneven: roughly 84 to 90% of developers use some AI coding tool, but only about 13% of teams run AI across the full development lifecycle and about 22% have deployed true autonomous coding agents; at large companies (5,000-plus employees), roughly 40% show platform-level adoption against 80%-plus who are experimenting. Read together, these confirm the pattern already flagged in the last review: near-universal individual tool use sits well ahead of organization-wide, governed deployment. Gartner's warning that over 40% of agentic AI projects will be canceled by the end of 2027, and GitLab's finding that most adopters have not solved maintainability, auditability and technical-debt risk, both still describe the gap firms are managing into. What is new is independent confirmation that the capability question (can an agent write working code) is settled at a scale most firms will recognize, while the governance and maintenance question remains the live one. For mid-size firms, the practical takeaway is unchanged from the last review: budget for who maintains, secures and audits a custom build for years, not just for who writes the first working version.
Updated 2026-11-17 · 4 sources
SoftHouse Advisory published a full evidence-checked brief on this stream on 26 October 2026, independently peer-reviewed before publication. The short version: AI coding is measurably changing software buying, but the evidence supports selective substitution and more competition, not an industry-wide revenue collapse. McKinsey (25 August 2026, 1,719 respondents, 97 countries) found 32% of organizations had declined at least one software purchase because they could build the functionality themselves. Retool (17 February 2026, 817 customers and builders, a self-selected sample) found 35% had replaced at least one SaaS tool with a custom build. Named vendor cases (Harmonic, Nursa) show real but partial substitution, still dependent on other paid platforms. Incumbent SaaS revenue continues to show no sign of broad collapse: ServiceNow's third-quarter 2026 results (quarter ended 30 September 2026), reported after this stream's last update, show subscription revenue of 3.988 billion USD, up 21% year over year, with full-year subscription guidance raised and its AI annual-contract-value target raised to 1.7 billion USD. Salesforce's most recent quarter also showed double-digit year-over-year subscription growth. Pricing is shifting toward paying for outcomes (Intercom's Fin agent) rather than per seat. The full brief, with named sources for every figure, is published as the first piece on this page.
Updated 2026-11-17 · 1 source
Measured data continues to show a real but narrow effect, concentrated on young and entry-level workers in routine roles, not an economy-wide wave of AI-driven layoffs. The Stanford Digital Economy Lab's ongoing ADP-payroll-based tracking remains the clearest primary measurement: employment for workers aged 22 to 25 in the most AI-exposed occupations continues to run meaningfully below the path expected if they had kept pace with less-exposed peers, a gap the researchers describe as widening through 2026 and continue to treat as an early, descriptive finding rather than a proven causal estimate. A separate industry index (S&P Global, surveyed mid-2026) reports that firms' own assessment of AI's net employment effect shifted over the prior twelve months from neutral-to-slightly-positive to net negative, a sentiment shift rather than a payroll measurement, and should be read as a leading indicator rather than confirmed job losses. Commentary from Goldman Sachs in late October and early November 2026 argues that AI need not cause mass layoffs to leave lasting damage, since displaced workers in a thin market for their specific skills can face long periods of underemployment even without a visible layoff spike; this remains analysis and forecasting, not a measured outcome. Employer-stated layoff reasons tracked by Challenger, Gray and Christmas through September 2026 continued to cite AI in a meaningful but not dominant share of announced cuts, and these are self-reported employer reasons, not an independently audited cause. For mid-size firms, the practical takeaway is unchanged from the last review: the measured pressure falls on hiring younger, entry-level staff into the most automatable roles well before it shows up as cuts to existing headcount, and the newer sentiment data suggests firms' own expectations are shifting in a more cautious direction even where payroll data has not yet moved.
Updated 2026-11-17 · 4 sources
Methodology: Third base-pull (prior pulls: 2026-06-24, 2026-10-26). Scores are comparative across the five tracks in THIS snapshot, not absolute capability percentages. Several dimension scores this cycle are marked estimated in score_rationale where sources disagree or no benchmark was located; those are directional, not calibrated. Microsoft 365 Copilot is scored as a product/distribution/governance layer over partner frontier models, not as a standalone model, consistent with prior snapshots. Produced by SoftHouse Advisory.