[{"data":1,"prerenderedAt":473},["ShallowReactive",2],{"page:index-navigation":3,"home:service-page:footer":9,"page:help-pages":244,"page:service-page":268,"technical-blog-page-content":314,"technical-blog-post-measuring-unit-test-quality-with-mutation-testing":341,"technical-blog-author-measuring-unit-test-quality-with-mutation-testing":317},{"helpBlocks":4},[5,6,7,8],"advice","projects","people","support",{"groupedServices":10},[11],{"title":12,"groups":13},"Services",[14,55,96,135,172,208],{"title":15,"slug":16,"icon":17,"subtitle":18,"services":19},"Strategy","strategy","lucide:compass","Shaping AI and digital transformation strategies that help organisations prioritise investment, define practical roadmaps, strengthen governance, and build the operating models needed to adopt change with confidence.",[20,27,34,41,48],{"slug":21,"title":22,"subtitle":23,"description":24,"image":25,"icon":26},"ai-digital-strategy","AI & Digital Strategy","Supporting organisations through AI adoption and digital transformation","Assessing where AI and digital can deliver the greatest value, building the business case for investment and developing a prioritised roadmap grounded in what your organisation can deliver at pace and at scale.","\u002Fimg\u002Fservice-page\u002Fgen-ai.jpg","lucide:sparkles",{"slug":28,"title":29,"subtitle":30,"description":31,"image":32,"icon":33},"ai-digital-delivery","AI & Digital Delivery","Delivering strategic roadmaps to for digital and AI-enabled change","Providing the engineering, data science and AI capability needed to take complex AI and digital programmes from validated strategy through to successful delivery, for organisations that need both the technical depth and the delivery structure to execute consistently.","\u002Fimg\u002Fservice-page\u002Freporting.jpg","lucide:map",{"slug":35,"title":36,"subtitle":37,"description":38,"image":39,"icon":40},"ai-native-operating-models","AI Native Operating Models","Designing operating models that embed AI into teams, services and delivery","Designing and delivering AI-native operating model transformations across organisational structure, processes, workforce and technology, combining hands-on AI delivery experience with consulting expertise to deliver transformations focused on operational reality.","\u002Fimg\u002Fservice-page\u002Fcloud-design.jpg","lucide:building-2",{"slug":42,"title":43,"subtitle":44,"description":45,"image":46,"icon":47},"ai-data-maturity","AI & Data Maturity","Assessing organisational readiness and capability across AI and data","Delivering independent, evidence-based assessments of AI and data capability across people, process, technology and governance, identifying the most significant capability gaps and building a structured, prioritised plan to address them.","\u002Fimg\u002Fservice-page\u002Fanalytics.jpg","lucide:bar-chart-3",{"slug":49,"title":50,"subtitle":51,"description":52,"image":53,"icon":54},"governance-risk-security","Governance, Risk & Security","Establishing controls and frameworks for responsible, secure AI adoption","Helping organisations manage governance, risk and security across AI and digital programmes, from establishing AI governance frameworks and responsible AI policies to regulatory compliance and cyber security advisory.","\u002Fimg\u002Fservice-page\u002Fcloud-security.jpg","lucide:shield-check",{"title":56,"slug":57,"icon":58,"subtitle":59,"services":60},"Engineering","engineering","lucide:cog","Building large-scale, complex software platforms - from enterprise legacy modernisation, to greenfield projects, we deliver solutions that automate, streamline and simplify processes, meeting the needs of today and the future.",[61,68,75,82,89],{"slug":62,"title":63,"subtitle":64,"description":65,"image":66,"icon":67},"software-development-company-uk","Software Development","Engineering complex, mission critical software platforms","Designing and delivering large-scale, complex software platforms for enterprise and public sector organisations across industries, from initial discovery and design through to deployment and long-term support.","\u002Fimg\u002Fservice-page\u002Fsoftware-development.jpg","lucide:code",{"slug":69,"title":70,"subtitle":71,"description":72,"image":73,"icon":74},"ai-assisted-engineering","AI-Assisted Engineering","Accelerating software delivery with AI-enabled engineering practices","Embedding AI tools and practices across engineering teams to accelerate delivery, improve code quality and build lasting AI capability, from AI-assisted code generation and review to AI-powered testing and deployment automation.","\u002Fimg\u002Fservice-page\u002Fai-assisted-software-development.jpg","lucide:cpu",{"slug":76,"title":77,"subtitle":78,"description":79,"image":80,"icon":81},"systems-integration","Systems Integration","Integrating systems to operate as a single framework","Connecting core business systems so they operate as a single, coherent framework, covering ERP, CRM, finance and reporting platforms, with real-time data accuracy and the audit trails needed to manage complex enterprise environments reliably.","\u002Fimg\u002Fservice-page\u002Fsystem-intergration.jpg","lucide:git-merge",{"slug":83,"title":84,"subtitle":85,"description":86,"image":87,"icon":88},"mobile-development","Mobile Development","Delivering performant, scalable and robust user-driven mobile applications","Delivering performant, scalable and user-driven mobile applications for enterprise and public sector organisations, from cross-platform apps and PWAs to native iOS and Android solutions built for complex operational environments.","\u002Fimg\u002Fservice-page\u002Fmobile-development.jpg","lucide:smartphone",{"slug":90,"title":91,"subtitle":92,"description":93,"image":94,"icon":95},"front-end-development-company","Front End Development","Building intuitive and scalable user-focused digital experiences","Designing and building intuitive, accessible and performant front end applications, from UX strategy and prototyping through to full delivery, for organisations where digital experience is central to how they serve customers and teams.","\u002Fimg\u002Fservice-page\u002Ffront-end-software-development.jpg","lucide:layout",{"title":97,"slug":98,"icon":99,"subtitle":100,"services":101},"Testing","testing","lucide:check-circle","Helping organisations assure quality across platforms, products and services - from defining test approaches to automating validation, performance assurance and security-focused testing.",[102,108,115,122,129],{"slug":103,"title":104,"subtitle":105,"description":106,"image":32,"icon":107},"testing-strategy","Test Strategy","Defining scalable test approaches aligned to delivery goals and risk","Defining, auditing and improving software testing strategies for enterprise and public sector organisations, covering functional and non-functional testing, automation, tooling and team capability in a single coherent framework.","lucide:clipboard-check",{"slug":109,"title":110,"subtitle":111,"description":112,"image":113,"icon":114},"automated-testing","Automated Testing","Building automated test suites to improve quality, speed and confidence","Designing and delivering automated testing frameworks across end-to-end, API and performance testing, built around your technology stack and embedded directly into delivery pipelines for reliable, maintainable coverage at pace.","\u002Fimg\u002Fservice-page\u002Fdevops.jpg","lucide:bot",{"slug":116,"title":117,"subtitle":118,"description":119,"image":120,"icon":121},"performance-load-testing","Performance & Load Testing","Validating system performance, resilience and scalability under demand","Validating system performance under real-world demand for organisations where reliability, scalability and speed are business-critical, from pre-release testing and bottleneck identification through to continuous performance testing in delivery pipelines.","\u002Fimg\u002Fservice-page\u002Fcloud-native.jpg","lucide:gauge",{"slug":123,"title":124,"subtitle":125,"description":126,"image":127,"icon":128},"ai-testing-and-validation","AI Testing & Validation","Testing AI systems for quality, safety, reliability and expected outcomes","Validating AI models and systems across accuracy, reliability and deployment readiness, from pre-deployment model evaluation and independent AI assurance through to production monitoring and LLM red teaming.","\u002Fimg\u002Fservice-page\u002Fprototype.jpg","lucide:flask-conical",{"slug":130,"title":131,"subtitle":132,"description":133,"image":53,"icon":134},"security-testing","Security Testing","Identifying vulnerabilities and strengthening confidence in system security","Identifying and addressing security vulnerabilities across software systems, infrastructure, development practices and AI systems, covering penetration testing, threat modelling, vulnerability assessment and LLM red teaming.","lucide:shield",{"title":136,"slug":137,"icon":138,"subtitle":139,"services":140},"Data","data","lucide:database","Turning data into a valuable asset - from building centralised data platforms to delivering clear, accessible reporting and analytics, we enable teams to make faster, more informed decisions based on trusted data.",[141,146,152,159,165],{"slug":142,"title":143,"subtitle":144,"description":145,"image":32,"icon":47},"data-visualisation-consultancy","Data Visualisation & Reporting","Delivering custom reporting for data-driven decision making","Designing and delivering interactive dashboards, custom reports and real-time analytics aligned to your KPIs, giving operational and strategic teams the visibility they need to make informed, data-driven decisions.",{"slug":147,"title":148,"subtitle":149,"description":150,"image":46,"icon":151},"data-analytics-company","Data Analytics","Uncovering actionable insights for data-driven transformation","Analysing large-scale, complex datasets to surface trends, correlations and forecasts that inform decisions across sales, supply chain, operations and strategic planning, from predictive analytics to big data analysis.","lucide:line-chart",{"slug":153,"title":154,"subtitle":155,"description":156,"image":157,"icon":158},"data-migration-services","Data Migration","Delivering large migrations across architectures and environments","Delivering large-scale, complex data migrations across cloud, database and application systems, ensuring data is clean, accurate and reliable throughout transition with no disruption to operations or loss of data fidelity.","\u002Fimg\u002Fservice-page\u002Fmigration.jpg","lucide:database-backup",{"slug":160,"title":161,"subtitle":162,"description":163,"image":164,"icon":138},"data-engineering-services","Data Engineering","Engineering pipelines to consolidate, process and aggregate data","Building the scalable data pipelines, storage architectures and integration frameworks that consolidate, process and aggregate data across the organisation, providing the foundations for effective analytics and decision making.","\u002Fimg\u002Fservice-page\u002Fdata-engineering.jpg",{"slug":166,"title":167,"subtitle":168,"description":169,"image":170,"icon":171},"data-platform-consulting","Enterprise Data Products","Designing, delivering and optimising enterprise data platforms to provide unified data foundations","Designing, implementing and optimising enterprise data platforms using Snowflake, Databricks and Apache Spark, delivering unified, governed data foundations that support analytics, self-service insights and AI capability at scale.","\u002Fimg\u002Fservice-page\u002Fenterprise-data-products.jpg","lucide:layers",{"title":173,"slug":174,"icon":175,"subtitle":176,"services":177},"AI","ai","lucide:brain","Guiding organisations through the practical application of AI - from early experimentation to production-grade systems, we create solutions that improve services and enable organisations to make decisions better, faster and at scale.",[178,183,190,195,202],{"slug":179,"title":180,"subtitle":181,"description":182,"image":127,"icon":128},"ai-consultancy","Prototypes and PoCs","Validating AI opportunities through rapid experimentation and prototyping","Validating AI concepts through rapid prototyping and proof of concept delivery across language and conversation, vision and detection, and prediction and optimisation use cases, helping organisations de-risk AI investment before committing to full development.",{"slug":184,"title":185,"subtitle":186,"description":187,"image":188,"icon":189},"ai-machine-learning-company","Production AI","Designing and delivering production-ready AI solutions at scale","Taking validated AI models into production for enterprise and public sector organisations, addressing model performance, integration, MLOps and ongoing monitoring to ensure AI systems perform consistently in live environments over the long term.","\u002Fimg\u002Fservice-page\u002Fconcept-to-delivery.jpg","lucide:rocket",{"slug":191,"title":192,"subtitle":193,"description":194,"image":25,"icon":114},"agentic-ai","Agentic AI","Building agent-based AI systems that can reason, act and automate","Building, running and supporting agentic AI systems for enterprise and public sector organisations, executing multi-step workflows, coordinating across systems and completing complex tasks with minimal human intervention.",{"slug":196,"title":197,"subtitle":198,"description":199,"image":200,"icon":201},"ai-machine-learning-operations-consulting","AI Cloud Platforms\u002FServices","Providing platforms and services to build, deploy and scale trusted AI","Building, deploying and scaling AI solutions across Azure, AWS and Google Cloud, from developing solutions using pre-built APIs and managed AI services through to foundation model customisation and cloud AI optimisation.","\u002Fimg\u002Fservice-page\u002Fai-cloud.jpg","lucide:cloud-cog",{"slug":203,"title":204,"subtitle":205,"description":206,"image":207,"icon":175},"ai-machine-learning-consulting","Data Science and ML","Building data science and machine learning solutions tailored to real-world challenges","Delivering data science and machine learning solutions for enterprise and public sector organisations across forecasting and prediction, insight and discovery, and optimisation and recommendation use cases.","\u002Fimg\u002Fservice-page\u002Fml.jpg",{"title":209,"slug":210,"icon":211,"subtitle":212,"services":213},"Cloud","cloud","lucide:cloud","Supporting organisations in building secure, scalable cloud environments - whether migrating systems, modernising platforms, or optimising existing infrastructure, we focus on delivering cloud solutions that improve performance, resilience, and cost-efficiency.",[214,220,227,233,239],{"slug":215,"title":216,"subtitle":217,"description":218,"image":39,"icon":219},"cloud-architecture-consulting","Cloud Architecture & Design","Designing cloud solutions optimised for resilience and scale","Designing and architecting robust, scalable cloud solutions across public and hybrid cloud environments, from cloud strategy and planning through to implementation and ongoing optimisation, using Well-Architected Frameworks and infrastructure-as-code techniques.","lucide:network",{"slug":221,"title":222,"subtitle":223,"description":224,"image":225,"icon":226},"cloud-migration-consulting","Cloud Adoption & Migration","Migrating to cloud infrastructures to enhance accessibility, performance and scalability","Delivering seamless migration of systems, data and workloads to optimal cloud platforms with minimal disruption to operations, across Azure, AWS and Google Cloud, from initial assessment and adoption planning through to post-migration optimisation and support.","\u002Fimg\u002Fservice-page\u002Fcloud-adoption.jpg","lucide:cloud-upload",{"slug":228,"title":229,"subtitle":230,"description":231,"image":120,"icon":232},"cloud-application-development","Cloud Native Solutions","Building scalable and resilient cloud native solutions rapidly","Designing and building highly scalable, resilient cloud native applications that fully harness the advantages of dynamic cloud environments, from microservices architecture and containerisation through to automated CI\u002FCD pipelines and continuous delivery.","lucide:boxes",{"slug":234,"title":235,"subtitle":236,"description":237,"image":113,"icon":238},"devops-consulting-services","DevOps & Pipelines","Embedding automation through DevOps for continuous integration, delivery and deployment","Transforming how software is delivered and released through optimised CI\u002FCD pipelines, infrastructure-as-code and DevOps culture, enabling teams to ship updates frequently, reliably and at pace across complex enterprise and public sector programmes.","lucide:workflow",{"slug":240,"title":241,"subtitle":242,"description":243,"image":53,"icon":54},"cloud-security-consulting","Cloud Security","Securing cloud environments through architecture, automation and optimisation","Securing cloud environments and workloads through risk-based security architecture, automated controls and continuous monitoring, helping organisations adopt cloud securely while meeting compliance obligations across Azure, AWS and Google Cloud.",[245,250,256,262],{"path":246,"title":247,"blockDescription":248,"icon":74,"bgColour":249},"\u002Fhelp\u002Fadvice","Advise","Offering strategic advice, technology consultancy and support on end-to-end digital transformation.","#1174d6",{"path":251,"title":252,"blockDescription":253,"icon":254,"bgColour":255},"\u002Fhelp\u002Fpeople","Collaborate","Augment your internal teams with skilled, experienced specialists to accelerate delivery and reduce risk.","lucide:users","#F4743B",{"path":257,"title":258,"blockDescription":259,"icon":260,"bgColour":261},"\u002Fhelp\u002Fprojects","Deliver","Combining domain and technology expertise to build digital products and services at speed and scale.","lucide:kanban-square","#348d69",{"path":263,"title":264,"blockDescription":265,"icon":266,"bgColour":267},"\u002Fhelp\u002Fsupport","Maintain","Managed support and hosting services to keep business-critical platforms secure, resilient and scalable.","mdi:gauge","#A93F55",{"groupedServices":269},[270],{"title":12,"groups":271},[272,279,286,293,300,307],{"title":15,"slug":16,"icon":17,"subtitle":18,"services":273},[274,275,276,277,278],{"slug":21,"title":22,"subtitle":23,"description":24,"image":25,"icon":26},{"slug":28,"title":29,"subtitle":30,"description":31,"image":32,"icon":33},{"slug":35,"title":36,"subtitle":37,"description":38,"image":39,"icon":40},{"slug":42,"title":43,"subtitle":44,"description":45,"image":46,"icon":47},{"slug":49,"title":50,"subtitle":51,"description":52,"image":53,"icon":54},{"title":56,"slug":57,"icon":58,"subtitle":59,"services":280},[281,282,283,284,285],{"slug":62,"title":63,"subtitle":64,"description":65,"image":66,"icon":67},{"slug":69,"title":70,"subtitle":71,"description":72,"image":73,"icon":74},{"slug":76,"title":77,"subtitle":78,"description":79,"image":80,"icon":81},{"slug":83,"title":84,"subtitle":85,"description":86,"image":87,"icon":88},{"slug":90,"title":91,"subtitle":92,"description":93,"image":94,"icon":95},{"title":97,"slug":98,"icon":99,"subtitle":100,"services":287},[288,289,290,291,292],{"slug":103,"title":104,"subtitle":105,"description":106,"image":32,"icon":107},{"slug":109,"title":110,"subtitle":111,"description":112,"image":113,"icon":114},{"slug":116,"title":117,"subtitle":118,"description":119,"image":120,"icon":121},{"slug":123,"title":124,"subtitle":125,"description":126,"image":127,"icon":128},{"slug":130,"title":131,"subtitle":132,"description":133,"image":53,"icon":134},{"title":136,"slug":137,"icon":138,"subtitle":139,"services":294},[295,296,297,298,299],{"slug":142,"title":143,"subtitle":144,"description":145,"image":32,"icon":47},{"slug":147,"title":148,"subtitle":149,"description":150,"image":46,"icon":151},{"slug":153,"title":154,"subtitle":155,"description":156,"image":157,"icon":158},{"slug":160,"title":161,"subtitle":162,"description":163,"image":164,"icon":138},{"slug":166,"title":167,"subtitle":168,"description":169,"image":170,"icon":171},{"title":173,"slug":174,"icon":175,"subtitle":176,"services":301},[302,303,304,305,306],{"slug":179,"title":180,"subtitle":181,"description":182,"image":127,"icon":128},{"slug":184,"title":185,"subtitle":186,"description":187,"image":188,"icon":189},{"slug":191,"title":192,"subtitle":193,"description":194,"image":25,"icon":114},{"slug":196,"title":197,"subtitle":198,"description":199,"image":200,"icon":201},{"slug":203,"title":204,"subtitle":205,"description":206,"image":207,"icon":175},{"title":209,"slug":210,"icon":211,"subtitle":212,"services":308},[309,310,311,312,313],{"slug":215,"title":216,"subtitle":217,"description":218,"image":39,"icon":219},{"slug":221,"title":222,"subtitle":223,"description":224,"image":225,"icon":226},{"slug":228,"title":229,"subtitle":230,"description":231,"image":120,"icon":232},{"slug":234,"title":235,"subtitle":236,"description":237,"image":113,"icon":238},{"slug":240,"title":241,"subtitle":242,"description":243,"image":53,"icon":54},{"id":315,"title":316,"author":317,"blogTags":317,"body":318,"customExcerpt":317,"date":317,"description":322,"excerpt":317,"extension":325,"h1":326,"image":317,"meta":327,"navigation":328,"path":329,"readingStats":330,"seo":335,"stem":339,"__hash__":340},"technicalBlogPage\u002Ftechnical-blog-page.md","Technical Blog Page",null,{"type":319,"value":320,"toc":321},"minimark",[],{"title":322,"searchDepth":323,"depth":323,"links":324},"",2,[],"md","Technical insights, stories and opinions from our team of consultants, analysts, developers and testers.",{},true,"\u002Ftechnical-blog-page",{"text":331,"minutes":332,"time":333,"words":334},"1 min read",0.29,17400,58,{"title":336,"description":337,"keywords":338},"Technology Insights | Audacia Digital Transformation","Technical insights, stories and opinions from leading software development company Audacia's teams of consultants, analysts, developers and testers.","Digital transformation technology, Technology blog, Technology insights, software development blog, software engineering blog","technical-blog-page","XD1gWfEur9Mu64WptQ5pF-e7Ej60LGiq7sRC58wOAeQ",{"id":342,"title":343,"author":317,"blogTags":344,"body":346,"customExcerpt":457,"date":458,"description":351,"excerpt":317,"extension":325,"h1":317,"image":459,"meta":460,"navigation":328,"path":461,"readingStats":462,"seo":467,"stem":471,"__hash__":472},"technicalBlog\u002Ftechnical-blog\u002Fmeasuring-unit-test-quality-with-mutation-testing.md","Measuring Unit Test Quality With Mutation Testing ",[97,345],"Software Testing ",{"type":319,"value":347,"toc":448},[348,352,355,360,363,366,370,373,380,383,387,390,393,396,400,403,409,412,415,419,422,425,428,432,435,439,442,445],[349,350,351],"p",{},"When a codebase shows 100% coverage, every line has been run by a test, so it seems safe to assume the code is properly checked. But code coverage only measures whether a line was executed during a test run, not whether that test would identify if the logic behind that line changed.",[349,353,354],{},"Mutation testing addresses this gap directly, by deliberately introducing small bugs into a codebase and checking whether the existing tests catch them. The following blog sets out why this distinction matters, how mutation testing works in practice and what it revealed when run against both a small example and a much larger, real production codebase.",[356,357,359],"h2",{"id":358},"why-code-coverage-isnt-enough","Why code coverage isn't enough",[349,361,362],{},"Code coverage is the percentage of code touched by a unit test. Tooling tracks which lines are hit while tests run and tallies this up. The idea is to use this to target areas that remain uncovered, particularly high-risk areas or ones that are hard to reach through automated testing.",[349,364,365],{},"The problem is that a high coverage percentage can create a false sense of security. We conducted a demonstration to test this: starting from an example project with 100% test coverage, a single line of code was deliberately changed to introduce a bug, in this case swapping which item in a set of products was treated as the cheapest for a discount calculation. Despite this change, every existing unit test still passed. In other words, a project with 100% code coverage did not have 100% protection against this kind of fault. At that point, only code review or manual and automation testing downstream would have any chance of catching it.",[356,367,369],{"id":368},"what-mutation-testing-actually-measures","What mutation testing actually measures",[349,371,372],{},"Mutation testing works by creating many altered versions of a codebase, known as mutants, each with a small change introduced. A mutant is considered \"killed\" if any unit test fails as a result of that change, and \"survived\" if all tests continue to pass despite the change.",[349,374,375],{},[376,377],"img",{"alt":378,"src":379,"title":378},"Mutation testing workflow","\u002Fimg\u002Ftechnical-blog\u002Fmutation-testing-flow-final.jpg",[349,381,382],{},"Rather than asking how much code has been executed by tests, mutation testing asks how much of that code, if changed incorrectly, would be caught by a failing test. A mutation score close to 100% means that changing almost any line of code causes a test to fail, which is a more meaningful measure of test quality than coverage alone.",[356,384,386],{"id":385},"running-mutation-testing-in-net","Running mutation testing in .NET",[349,388,389],{},"Mutation testing has existed for some time, though it has been available in .NET for a shorter period. The tool used here was Stryker.NET, installed and run as a .NET tool. Where unit tests span multiple projects, the tool needs to be pointed at the specific project reference containing the code under test, in this case a domain layer project.",[349,391,392],{},"Stryker.NET works by scanning code for specific patterns and substituting them with alternatives: an addition might be swapped for a subtraction, a greater-than comparison for a less-than, and, relevant to the example below, a LINQ OrderBy call for OrderByDescending, or vice versa. Mutants that fail to compile are discarded.",[349,394,395],{},"The tool produces an HTML report by default, showing which mutants were created, which were killed and which survived, alongside the resulting mutation score. It also supports JSON and Markdown output, which could be used for build pipeline reporting or posting a summary to a pull request.",[356,397,399],{"id":398},"a-worked-example","A worked example",[349,401,402],{},"Running the tool against the small example project generated seven mutants. Six were killed and one survived, giving a mutation score of 6\u002F7, or 85.71%. The surviving mutant pinpointed the deliberately introduced bug: the swap between OrderBy and OrderByDescending in the discount calculation, which none of the existing tests were sensitive enough to detect.",[349,404,405],{},[376,406],{"alt":407,"src":408,"title":407},"Mutant code example","\u002Fimg\u002Ftechnical-blog\u002Ftech-blog-mutation-testing-3.jpg",[349,410,411],{},"To fix this, an additional test case was added, using products with three distinct prices rather than identical ones, so that the ordering of items by price would affect the result. After removing the deliberately introduced bug and adding this test, the mutation score on a second run reached 100%, with all seven mutants killed, including the one that had previously survived.",[349,413,414],{},"A 100% mutation score is highly unlikely to be realistic on a larger, real-world codebase. It was only achievable here because the example was narrow and self-contained, testing a single, well-defined piece of logic.",[356,416,418],{"id":417},"testing-at-scale","Testing at scale",[349,420,421],{},"We also ran the tool against one of our larger internal production codebases, which has around 5,000 unit tests. The initial full run returned a mutation score of around 40%. In other words, the existing tests caught around four in ten of the deliberate changes made to the code.",[349,423,424],{},"This score isn't the same as code coverage. A low score doesn't automatically mean more tests are needed. It can just as easily mean some existing tests aren't really checking anything, and those are better improved or removed than simply added to.",[349,426,427],{},"So rather than treating the score as a target to push towards 100%, we recommend using it as a map: it shows where tests are doing real work, where they add runtime, and where extra effort would pay off.",[356,429,431],{"id":430},"what-makes-a-unit-test-valuable","What makes a unit test valuable",[349,433,434],{},"A recurring theme in these tests was that a unit test which never fails, regardless of what changes are made to the underlying code, is not adding much value. The test may simply assert that an operation was successful in a general sense, without checking the specific outcome the code is meant to produce. This kind of test can catch runtime errors, but it won't catch a business logic mistake.",[356,436,438],{"id":437},"conclusion","Conclusion",[349,440,441],{},"Mutation testing offers a more representative view of test suite quality than code coverage alone, because it tests whether unit tests would catch a real fault, rather than whether a line of code was executed.",[349,443,444],{},"The trade-off is runtime: at the scale of a codebase with tens of thousands of mutants, a full mutation testing run can take hours.",[349,446,447],{},"Since this test was carried out, Stryker.NET has introduced a preview test runner built on the Microsoft Testing Platform, designed to remove the repeated start-up overhead that slows down large mutation testing runs. This may close the gap for large codebases, and it's a sign that runtime constraint is likely to ease over time rather than being a fixed limitation of the tool. The new runner is still maturing, though, so it's worth checking its current status before relying on it.",{"title":322,"searchDepth":323,"depth":323,"links":449},[450,451,452,453,454,455,456],{"id":358,"depth":323,"text":359},{"id":368,"depth":323,"text":369},{"id":385,"depth":323,"text":386},{"id":398,"depth":323,"text":399},{"id":417,"depth":323,"text":418},{"id":430,"depth":323,"text":431},{"id":437,"depth":323,"text":438},"A codebase with 100% code coverage can still let a simple logic bug through, because coverage only shows which lines ran during a test, not whether a test would notice if they changed. This piece looks at why coverage falls short as a single measure of test quality, how mutation testing with Stryker.NET exposes weak tests and what a run against a production codebase of around 5,000 unit tests revealed.","2026-10-16T08:21:40.701Z","\u002Fimg\u002Ftechnical-blog\u002Fmutation-testing-cover.jpg",{},"\u002Ftechnical-blog\u002Fmeasuring-unit-test-quality-with-mutation-testing",{"text":463,"minutes":464,"time":465,"words":466},"7 min read",6.135,368100,1227,{"title":468,"description":469,"keywords":470},"Mutation Testing and Unit Test Quality | Insights","Code coverage shows which lines ran, not whether tests would catch a bug. Learn how mutation testing with Stryker.NET gives a truer measure of unit test quality.","mutation testing, Stryker.NET, mutation score, unit test quality, code coverage vs mutation testing, what is mutation testing, how to measure unit test quality, is 100% code coverage enough","technical-blog\u002Fmeasuring-unit-test-quality-with-mutation-testing","gfn5KW_3KULmApGVPW5pt5B_Z4lC2MaMx_9y1xoEsS4",1792144213493]