At Stanford, the founders of the Diffbot startup specializing in machine learning have developed a system for tracking companies applying sanctions. As a result, individuals and businesses can stay up to date with developments concerning the affected companies.
“Using natural language processing, Diffbot scans the public network and structures it into a public Knowledge Graph database, constantly supported by an artificial intelligence system,” said Diffbot CEO Mike Tung.
As Tung noted, attempts to create such lists have been made repeatedly. However, the lists compiled by the media contain many errors. Similarly, community projects such as opensanctions.org, historically focused on aggregating official government lists, have had a hard time coping with the flow of messages.
Knowledge Graph Diffbot scans the entire open Internet, including Russian-language pages. Using Natural Language Processing, the system reads pages in the native language. To check whether a source is trustworthy or not, Diffbot assigns a rating to each domain and combines information from various sources.
“To verify the reliability of information, algorithms assign each source a confidence score based on knowledge, as well as a probability score for each fact obtained as a result of this process. When searching in Knowledge Graph, clients find a set of facts with a probability level higher than the default one,” the expert said.
The algorithms take into account whether a particular fact appears in several different sources, the historical track record of facts created by this source over time, the organization that owns the site, and whether the fact is consistent with data from other sources.
“In order to track sanctions in real time, we display sanctions appearing in at least three independent, reliable sources,” Tung said.
At the time of writing, more than 600 brands were listed in the Diffbot sanctions tracker.