{"id":1574,"date":"2021-04-23T08:55:19","date_gmt":"2021-04-23T06:55:19","guid":{"rendered":"https:\/\/innovatrics.com\/trustreport\/?p=1574"},"modified":"2023-06-15T14:22:52","modified_gmt":"2023-06-15T12:22:52","slug":"how-independent-testing-has-impacted-face-recognition-and-helped-create-trustworthy-marketing","status":"publish","type":"post","link":"https:\/\/innovatrics.com\/trustreport\/how-independent-testing-has-impacted-face-recognition-and-helped-create-trustworthy-marketing\/","title":{"rendered":"How independent testing has impacted face recognition accuracy\u2013 and helped create trustworthy marketing"},"content":{"rendered":"\n
\"\"<\/figure>\n\n\n\n
\n
\n
\n\n\n\n

Contributors<\/h4>\n\n\n
\n \"\"\n
\n

J\u00e1n Lunter<\/h3>\n

CEO of Innovatrics<\/span><\/p>\n <\/div>\n<\/div>\n\n

\n \"\"\n
\n

Mat\u00fa\u0161 Kapusta<\/b><\/span><\/span><\/h3>\n

Head of Government Solutions<\/span><\/span><\/p>\n <\/div>\n<\/div><\/div>\n\n\n\n

\n

Does a generic claim like \u201c99.99% face recognition accuracy\u201d<\/em> really make the grade now, if not supported by evidence from independent assessments?<\/strong><\/strong><\/strong><\/p>\n<\/div>\n<\/div>\n\n\n\n

\n
<\/div>\n\n\n\n
\n

Algorithms responding to current challenges<\/h2>\n\n\n\n

The FRVT\u00a0(Face\u00a0Recognition Vendor Test)\u00a0evaluation is both a training ground and a somewhat\u00a0cruel arena. It provides reliable benchmarking for face recognition accuracy in both verification and identification, and it confronts biometric algorithms with additional challenges\u00a0in order to\u00a0check how they perform\u00a0in different circumstances.\u00a0<\/p>\n\n\n\n

The result is a list of contenders,\u00a0ranked from the lowest to the highest performance. All data\u00a0is\u00a0public and accessible through the NIST website.<\/p>\n<\/div>\n<\/div>\n\n\n\n

\n
\n
\n\n\n\n
Brief history of FRVT<\/strong><\/h6>\n\n\n\n

In the early 2000s, the National Institute of Standards and Technology (NIST) in the U.S. decided to start independent evaluations for facial recognition technologies. The federal agency wanted to provide the U.S. government with a benchmark for the best available technology in the field, both in the market and as a prototype. At that time, facial recognition was just starting to show its potential. This is how the FRVT (Face Recognition Vendor Test) was born.<\/p>\n\n\n\n

The first large-scale FRVT evaluations took place during the 2000s. The algorithms\u2019 accuracy was challenged on a standard image database, called FERET, which at that time consisted of 2,413 still facial images representing 856 people. Today, FRVT has become an ongoing program. At the moment, approximately 200 algorithms have been submitted by about a hundred universities, research institutes, and companies from all over the world.<\/p>\n<\/div>\n\n\n\n

\n

Tests like FRVT were born\u00a0in order\u00a0to provide an authoritative standard in the evaluation of new biometric technologies. Together with other NIST evaluation programs like FpVTE\u00a0(for\u00a0facial recognition and fingerprints) and MINEX\u00a0(which assesses the capacity of an algorithm to be used by different vendors), they have\u00a0resulted in the fast-paced biometrics industry\u00a0taking on an interesting dimension. <\/p>\n\n\n\n

This is now a community\u00a0in which\u00a0the capability to steadily improve your own algorithms\u00a0is\u00a0what really matters.\u00a0<\/p>\n\n\n\n

Marketing has its consequences too. Does a generic claim like\u00a0\u201c99.99%\u00a0face recognition accuracy\u201d really make the grade now, if not supported by evidence from independent assessments?\u00a0Not really.\u00a0<\/p>\n\n\n\n

Yet this\u00a0presents\u00a0a new challenge to biometrics companies participating in independent evaluations:\u00a0how to\u00a0communicate\u00a0facts and rankings\u00a0to their customers, that\u00a0are not always easy to digest.<\/p>\n\n\n\n

\u201cParticipating in the FRVT shows transparency and confidence in the product.\u201d<\/strong><\/p><\/blockquote>\n\n\n\n

We discussed the value of independent face recognition accuracy testing for biometrics companies with J\u00e1n Lunter, CEO at Innovatrics, and the company’s Head of Government Solutions, Mat\u00fa\u0161 Kapusta.<\/h3>\n\n\n\n

<\/p>\n<\/div>\n<\/div>\n\n\n\n

\n
<\/div>\n\n\n\n
\n

How has independent testing impacted your work at Innovatrics?<\/strong><\/p>\n\n\n\n

J. Lunter:<\/strong> It helped us immensely, from the very start of our story. At the beginnings of Innovatrics we were mainly focusing on fingerprint recognition. NIST set up a benchmark for fingerprint algorithms and we decided to participate. We may have been a small company among giants, such as NEC, but we had extremely fast and accurate algorithms, which were also not too demanding on hardware.<\/p>\n\n\n\n

This independent comparison helped us in many international tenders in which we competed, and won, against much larger companies. Independent testing provided by NIST really does create a level playing field: you can challenge the biggest companies on equal terms. And that\u2019s terribly exciting!<\/p>\n\n\n

\n
\n
\n \"\"\n\n <\/div>\n
\n
\n Read more<\/a>\n

\n \n How the accuracy of face-recognition technology has improved over time <\/a>\n <\/h2>\n <\/div>\n\n\n <\/div>\n <\/div>\n<\/div>\n<\/div>\n<\/div>\n\n\n\n
\n
\n
\n\n\n\n

The Innovatrics algorithm has scored top marks in accuracy and speed. These are crucial for large-scale applications such as document verification, criminal investigation and border control. Read more about\u00a0the\u00a0results\u00a0here<\/a>.\u00a0<\/p>\n<\/div>\n\n\n\n

\n

What is your experience with the Facial Recognition Vendor Test (FRVT)?<\/strong><\/p>\n\n\n\n

J. Lunter:<\/strong> When we turned to the FRVT our goal was not to merely participate, but to consistently come in among the top 20 or so. This is one of the priorities of our research team. FRVT provides a wealth of data about all our algorithms: we can see where we are doing well and where we lag behind and need to get better.<\/p>\n\n\n\n

M. Kapusta:<\/strong> The development in facial recognition is extremely fast. It\u2019s impressive; even if you have a top algorithm in a certain category, within half a year that same algorithm would end up somewhere in the middle of the table\u2026 or worse.<\/p>\n\n\n\n

The reports provided by NIST are extremely detailed and precise. We can study them to see exactly what we need to do in order to either catch up with the market or to improve our position in the fields in which we\u2019re already leading. For example, we are one of the few companies that are actually able to verify children\u2019s faces. Children\u2019s faces are smaller and change faster, so they provide a bigger challenge for algorithms.<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

How do you use FRVT results to market your technology?<\/strong><\/p>\n\n\n\n

J. Lunter:<\/strong> For us, the participation in FRVT and in other independent testing provided by NIST shows transparency and confidence in what we are doing. If you offer facial recognition and you are not in the FRVT comparisons, you\u2019ll be having a difficult time in persuading customers to opt for your offer.<\/p>\n\n\n\n

This helps us when dealing with certain customers, who sometimes are in contact with vendors that offer impossible numbers but which cannot be found in FRVT rankings at all. Most of the companies that take facial recognition seriously are in the benchmark. And, of course, being among the top competitors is great marketing.<\/p>\n\n\n\n

M. Kapusta: <\/strong>You also have to know how to communicate the results of the FRVT to customers, so that they can really focus on what matters most to them. If you are not deeply involved in biometrics, the FRVT reports are anything but easy to read and interpret.<\/p>\n\n\n\n

So, every time a new report from NIST comes out, we extract the information that is relevant for us and for our customers. For example, comparing us with the largest ABIS providers shows that we are one of the best in the world, as some of the top-ranking algorithms are actually from universities or research labs and they are not used in commercial products.<\/p>\n\n\n\n

We also split data into use-cases, to show how we perform in small databases, for example in attendance systems within companies, and in large ones, such as criminal or border control databases.<\/p>\n\n\n\n

Are you going to keep engaging in FRVT and other independent testing in the future?<\/strong><\/p>\n\n\n\n

J. Lunter:<\/strong> Definitely. And it is great that other emergent technologies are getting a similar independent testing treatment. Recently we passed tests by iBeta \u2013 an independent biometrics testing lab \u2013 with our liveness check technology, and we are one of the first in the world to actually do so. These evaluations compare liveness solutions from different companies and check whether they can be spoofed by showing a photo or a mask. We are now preparing for level 2 of these tests.<\/p>\n\n\n\n

<\/p>\n<\/div>\n<\/div>\n\n\n\n

\n
\n
\n\n\n\n
\"\"<\/figure>\n\n\n\n

Blair Crawford<\/strong><\/strong> is the co-founder and managing Director of Daltrey<\/a>, an Australian biometric identity provider with a mission to redefine how identity is used to create safer, more secure environments across government, critical infrastructure and enterprise. The Daltrey solution runs Innovatrics\u2019 algorithms, of which the face recognition accuracy performance is publicly visible on the NIST website.<\/p>\n\n\n\n

<\/div>\n\n\n\n
\n\n\n\n

Why having a safe global digital identity is important, how is its adoption faring and what will it look like in the near future?<\/strong>
Watch Blair Crawford\u2019s talk at Trust Report conference
Watch<\/a><\/p>\n<\/div>\n\n\n\n

\n
\n

Blair Crawford:
\u201cYou should never mark your own homework.\u201d<\/h2>\n\n\n\n

Opinion of a provider of a biometric identification platform<\/em><\/strong><\/p>\n\n\n\n

\u201cIf you are a good biometric provider, or anyone who operates in the security industry, you know that there is such a high level of competitiveness that you actually want to showcase results which can be reviewed and matched against others. Competition is good and it drives innovation. You are never going to get innovation if you are only using your own team to test what you built. Briefly said: you should never mark your own homework.<\/p>\n\n\n\n

If you are not willing to communicate with your customers about what they should expect from a security risk and user-experience perspective, then something is probably not stacking up in terms of your offering. On the other side, I think it is extremely important that the market has the opportunity to ask vendors for the details and the expectations for the services they are paying for.\u201d<\/p>\n<\/div><\/div>\n<\/div>\n<\/div>\n\n\n\n

\n\n\n\n
<\/div>\n\n\n\n
\n\n\n\n

Author: Giovanni Blandino<\/p>\n","protected":false},"excerpt":{"rendered":"

Does a generic claim like \u201c99.99% accuracy\u201d really make the grade now, if not supported by evidence from independent assessments?<\/p>\n","protected":false},"author":6,"featured_media":1773,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[1],"tags":[40,56,32],"yoast_head":"\nHow independent testing has impacted face recognition accuracy<\/title>\n<meta name=\"description\" content=\"Does a claim like \u201c99.99% face recognition accuracy\u201d really make the grade now, if not supported by evidence from independent assessments?\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/innovatrics.com\/trustreport\/how-independent-testing-has-impacted-face-recognition-and-helped-create-trustworthy-marketing\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"How independent testing has impacted face recognition accuracy\" \/>\n<meta property=\"og:description\" content=\"Does a generic claim like \u201c99.99% accuracy\u201d really make the grade now, if not supported by evidence from independent assessments?\" \/>\n<meta property=\"og:url\" content=\"https:\/\/innovatrics.com\/trustreport\/how-independent-testing-has-impacted-face-recognition-and-helped-create-trustworthy-marketing\/\" \/>\n<meta property=\"og:site_name\" content=\"Innovatrics\" \/>\n<meta property=\"article:published_time\" content=\"2021-04-23T06:55:19+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2023-06-15T12:22:52+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/innovatrics.com\/trustreport\/wp-content\/uploads\/2021\/04\/How-accuracy.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1200\" \/>\n\t<meta property=\"og:image:height\" content=\"631\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\">\n\t<meta name=\"twitter:data1\" content=\"7 minutes\">\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebSite\",\"@id\":\"https:\/\/innovatrics.com\/trustreport\/#website\",\"url\":\"https:\/\/innovatrics.com\/trustreport\/\",\"name\":\"Innovatrics\",\"description\":\"Len \\u010fal\\u0161ia WordPress str\\u00e1nka\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":\"https:\/\/innovatrics.com\/trustreport\/?s={search_term_string}\",\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"ImageObject\",\"@id\":\"https:\/\/innovatrics.com\/trustreport\/how-independent-testing-has-impacted-face-recognition-and-helped-create-trustworthy-marketing\/#primaryimage\",\"inLanguage\":\"en-US\",\"url\":\"https:\/\/innovatrics.com\/trustreport\/wp-content\/uploads\/2021\/04\/thumbnail-how-independent.png\",\"width\":1500,\"height\":1448},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/innovatrics.com\/trustreport\/how-independent-testing-has-impacted-face-recognition-and-helped-create-trustworthy-marketing\/#webpage\",\"url\":\"https:\/\/innovatrics.com\/trustreport\/how-independent-testing-has-impacted-face-recognition-and-helped-create-trustworthy-marketing\/\",\"name\":\"How independent testing has impacted face recognition accuracy\",\"isPartOf\":{\"@id\":\"https:\/\/innovatrics.com\/trustreport\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/innovatrics.com\/trustreport\/how-independent-testing-has-impacted-face-recognition-and-helped-create-trustworthy-marketing\/#primaryimage\"},\"datePublished\":\"2021-04-23T06:55:19+00:00\",\"dateModified\":\"2023-06-15T12:22:52+00:00\",\"author\":{\"@id\":\"https:\/\/innovatrics.com\/trustreport\/#\/schema\/person\/aa8ac119593dd2e3e7e0e0fe172a40fc\"},\"description\":\"Does a claim like \\u201c99.99% face recognition accuracy\\u201d really make the grade now, if not supported by evidence from independent assessments?\",\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/innovatrics.com\/trustreport\/how-independent-testing-has-impacted-face-recognition-and-helped-create-trustworthy-marketing\/\"]}]},{\"@type\":\"Person\",\"@id\":\"https:\/\/innovatrics.com\/trustreport\/#\/schema\/person\/aa8ac119593dd2e3e7e0e0fe172a40fc\",\"name\":\"Kristina Sebejova\",\"image\":{\"@type\":\"ImageObject\",\"@id\":\"https:\/\/innovatrics.com\/trustreport\/#personlogo\",\"inLanguage\":\"en-US\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/3f3afee03f3588bdfd390a79fbeea359?s=96&d=mm&r=g\",\"caption\":\"Kristina Sebejova\"}}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","_links":{"self":[{"href":"https:\/\/innovatrics.com\/trustreport\/wp-json\/wp\/v2\/posts\/1574"}],"collection":[{"href":"https:\/\/innovatrics.com\/trustreport\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/innovatrics.com\/trustreport\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/innovatrics.com\/trustreport\/wp-json\/wp\/v2\/users\/6"}],"replies":[{"embeddable":true,"href":"https:\/\/innovatrics.com\/trustreport\/wp-json\/wp\/v2\/comments?post=1574"}],"version-history":[{"count":76,"href":"https:\/\/innovatrics.com\/trustreport\/wp-json\/wp\/v2\/posts\/1574\/revisions"}],"predecessor-version":[{"id":4429,"href":"https:\/\/innovatrics.com\/trustreport\/wp-json\/wp\/v2\/posts\/1574\/revisions\/4429"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/innovatrics.com\/trustreport\/wp-json\/wp\/v2\/media\/1773"}],"wp:attachment":[{"href":"https:\/\/innovatrics.com\/trustreport\/wp-json\/wp\/v2\/media?parent=1574"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/innovatrics.com\/trustreport\/wp-json\/wp\/v2\/categories?post=1574"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/innovatrics.com\/trustreport\/wp-json\/wp\/v2\/tags?post=1574"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}