IBM exec said firm's new tool is like a 'nutrition label' for AI, providing detailed information to help flag bias or inaccuracy

  • Rob Thomas, IBM's head of AI, says a new system called AI Factsheets, would give a snapshot summary of AI programs,  offering information about the data used to build the program and how it was developed.
  • "The analogy I would use is this is the nutrition label for your AI,"  he told Business Insider. "You know how you go to the grocery store, you look at the box for what's actually the nutrition of what's in the box."

  • AI Factsheets is meant to change the view of AI-powered tools as opaque systems that can't always be trusted.
  • "We've seen lots of questions that were asked about face recognition," IBM Research fellow Saska Mojsilovic told Business Insider. "We've seen a lot of questions that are asked about the hiring systems. We've seen a lot of questions with credit approval. You have now this ability to understand how this application was developed.
  • Visit Business Insider's homepage for more stories.

Although the use of artificial intelligence in enterprise tech is growing, AI-powered tools can sometimes be opaque, without enough context on how the system works or why it made the decisions or recommendations that it did. 

IBM is looking to fix that with a new system called AI Factsheets, which was launched last week and which will offer a snapshot summary of an AI program deployed in a product, including the source of the data used to develop it and how the data was handled.

"The analogy I would use is, 'This is the nutrition label for your AI,'" Rob Thomas, head of IBM's AI business, told Business Insider. "You know how you go to the grocery store, you look at the box for what's actually the nutrition of what's in the box."

The head of IBM's Foundations of Trusted AI team who helped develop Factsheets, Saska Mojsilovic, said the tech giant came up with the system because "there were so many cases of AI that were not deployed correctly."

"We've seen lots of questions that were asked about face recognition," she said. "We've seen a lot of questions that are asked about the hiring systems. We've seen a lot of questions with credit approval. You have now this ability to understand how this application was developed. What was the purpose? What are the kinds of things that it should not be used for? Did it perform well when it was put in production? Et cetera."

In many cases, she said, the questions are about the data used to develop the AI program: "What did you do with that data? Did you manipulate it correctly? What are the metrics you used to judge that this model is accurate or that it's robust or that it is fair?"

If bias or inaccuracy is baked into the source data used to train an AI system, the results of that system will reflect that bias or inaccuracy. Researchers have found many instances where algorithms are biased against people of color or women.

These questions can be especially critical in industries that are subject to strict regulatory rules. Thomas cited the example of banks which are required to ensure that their policies are fair and nondiscriminatory.

"The source of that data is pretty important," he said. "Can you actually certify where the data came? Can you assert that there is no bias inherent in the models?"

The IBM AI Factsheets program underlines the need to strip away the "black box" nature of AI systems to achieve so-called "explainable AI," that would make it easier for managers and rank-and-file staff to use the technology effectively.

Thomas said he expects the premise of AI Factsheets to become a standard in the tech industry. "I wouldn't be surprised if you look a year or two out and this notion of a nutrition label for AI becomes default in the world, as opposed to something that's an exception," he said.

The Factsheets rollout fits into IBM's aggressive push, under CEO Arvind Krishna, to tap its massive AI capabilities to offer more products to customers. IBM also recently took a high-profile position on facial recognition: In June, in the wake of protests following the killing of George Floyd, Krishna announced that the company would stop offering facial recognition software and called for a national dialogue on the use of technology in law enforcement.

The company unveiled other AI products last week as well, including voice and reading tools that would enable a user to quickly find answers from a trove of documents.

"It's about pinpointing an answer in a big document," Salim Roukos, IBM's global leader for language research, told Business Insider. "For example, if you have a rental contract and you ask, 'Are dogs and cats allowed?' it will go read the contract and find a snippet that will answer your question."

Got a tip about IBM or another tech company? Contact this reporter via email at [email protected], message him on Twitter @benpimentel or send him a secure message through Signal at (510) 731-8429. You can also contact Business Insider securely via SecureDrop.

Source: Read Full Article