NIST releases AI engagement plan in response to Trump executive order

The U.S. Department of Commerce’s National Institute of Standards and Technology (NIST) released a draft of a plan for the federal creation of AI standards. The plan aims to meet the economic and national security needs of the United States and help the country maintain global dominance.

“U.S. engagement in establishing AI standards is critical; AI standards developed without the appropriate level and type of involvement of U.S. interests may exclude or disadvantage U.S.-based companies in the marketplace, as well as government agencies,” the draft report (PDF) reads. “Moreover, due to the foundational nature of standards, the lack of U.S. stakeholder engagement in the development of AI standards can negatively impact the innovativeness and competitiveness of U.S. interests in the long term.”

The document, titled “U.S. Leadership in AI: Plan for Federal Engagement in Developing Technical Standards and Related Tools, ” was put together in response to a Trump executive order mandating the creation of a federal AI engagement plan within 180 days. NIST will accept public comment on the draft document from members of the public until July 19 ahead of delivering a final document to the White House by August 10, a NIST spokesperson told VentureBeat in an email.

Standards efforts that “merit strong federal government engagement” include privacy protections, intent of design declarations from developers, and ensuring that AI is non-discriminatory and open and transparent, according to the report.

The government also seeks to “encourage input reflecting diverse communities of users and developers, vendors, and experts representing technical disciplines, as well as non-traditional disciplines of special importance to AI, such as ethicists, economists, legal professionals, and policy makers,” according to the draft. In other words, NIST wants to ensure all interested parties have a “seat at the table” and can help craft a policy of inclusion and fairness.

Though the report refers to AI bias, trustworthiness, and inclusion, it makes no explicit mention of any racial or gender group.

Research this year by both UC Berkeley and MIT found that algorithmic bias in housing lending disproportionately impacts Latinx and African-American households, and facial recognition systems from large tech companies work best for white men and worst for women of color, young people, and people who do not conform to gender roles.

The report outlines future steps, calling for the National Science and Technology Council (NSTC) Machine Learning/Artificial Intelligence (ML/AI) Subcommittee to create a standards coordinator and for the identification of research necessary to advance trust in AI. The report also recommends the creation of more tools to audit the performance of AI models, data sets, and test beds.

The document identifies standards efforts currently being undertaken by groups like IEEE and DARPA’s Explainable AI initiative, as well as existing benchmark data sets, like ImageNet. It also proposes tracking AI standards that are being developed by foreign governments and organizations.

NIST’s Facial Recognition Vendor Test is considered the gold standard by some, but others challenge its validity. In an interview with VentureBeat earlier this year, an AI researcher analyzing the history of NIST called the group unfit to shape AI standards — in part because it doesn’t employ AI ethicists and because it uses images in its Facial Recognition Vendor Test without user consent.

NIST is scheduled to produce a statistical analysis of facial recognition software performance based on race, gender, and other demographic types this fall, NIST Information Technology Laboratory director Charles Romine said in Congressional testimony last month. AI ethics recommendations for the U.S. Department of Defense are also due out this fall.

The White House Office of Science and Technology (OST) recently released updates to the national AI research and development strategy, which is meant to shape billions in federal AI research spending. The updates reinstate a number of Obama administration directives to federal agencies like the U.S. Department of Defense, the National Institutes of Health, and the National Science Foundation. In the update, the OST also called for more federal cooperation with private industry and international allies.

Like the updated AI research strategy, the federal engagement draft released today stresses work with international parties to “champion U.S. AI standards priorities in international AI standards development activities.”