Difference between revisions of "IKRIS"

Revision as of 02:26, 8 May 2021

MITRE Report on IKRIS Technology Transfer - John F. Sowa
From: http://www.jfsowa.com/ikl/IKLmitre.doc

According to The MITRE TECHNICAL REPORT  MTR060158,
MITRE Support to IKRIS Final Report by Brant A. Cheikes, Ph.D. from November 2006

Abstract

In November 2004, the Intelligence Community’s Advanced Research and Development Activity (ARDA), which subsequently became the Disruptive Technology Office (DTO), requested that the Northeast Regional Research Center (NRRC) hosted at MITRE provide technical oversight and management of a newly-funded “Challenge Workshop” called  IKRIS:  Interoperable Knowledge Representation for Intelligence Support. 
The IKRIS workshop was chartered to address the following challenge problems: 
(1) how to enable interoperability of knowledge representation (KR) technology developed by multiple organizations in multiple ARDA programs and designed to perform different tasks, and
(2) how to practically represent knowledge that is relevant to intelligence analysis tasks in a form that enhances automated support for analysts. 
This is MITRE’s final report on its activities and accomplishments as NRRC Program Manager for IKRIS.

The major accomplishments of the IKRIS Challenge Workshop are summarized as follows:

• IKL—the IKRIS Knowledge Language. This is the key technical result of the IKRIS Challenge Workshop. IKL is a formally-specified language, based on an emerging ISO standard called Common Logic, into and out of which a variety of distinctly different knowledge representation (KR) formalisms can be translated.
• ICL—the IKRIS Context Logic. ICL is a logic formalism for representing and reasoning about context-dependent knowledge, including alternative hypotheses, points of view, world states and scenarios.
• ISIT—the IKRIS Scenarios Inter-Theory. The Scenarios Inter-Theory specifies an approach to translating among the principal formalisms in current use for declaratively representing processes.
• Evaluation Report. The Evaluation Working Group has produced a report showing that IKL is a sound and effective mechanism for knowledge interchange.
• Capstone Demonstration. The Capstone Demonstration serves both as an evaluation of knowledge interchange using IKL, and as an illustration of the potential Intelligence Community impact of the IKRIS-developed approach to interoperability.
• IKL translators. IKRIS participants at Stanford University implemented a set of software tools for building automated translators into and out of IKL.

• Chapter 1 of this report documents the workshop’s milestones, participants, accomplishments and impact.
• Chapter 2 describes MITRE’s support activities and accomplishments, and
• Chapter 3 presents conclusions, recommendations and lessons learned.

Table of Contents

1 Introduction (pg 1)

In November 2004, the Intelligence Community’s Advanced Research and Development Activity (ARDA), which subsequently became the Disruptive Technology Office (DTO), requested that the Northeast Regional Research Center (NRRC) hosted at MITRE provide technical oversight and management of a newly-funded “Challenge Workshop” called IKRIS:  Interoperable Knowledge Representation for Intelligence Support.  The IKRIS workshop was chartered to address the following challenge problems:  (1) how to enable interoperability of knowledge representation (KR) technology developed by multiple organizations in multiple ARDA programs and designed to perform different tasks, and (2) how to practically represent knowledge that is relevant to intelligence analysis tasks in a form that enhances automated support for analysts.

According to ARDA’s workshop plan, MITRE was to serve as prime contractor for the effort, and was to subcontract with an approved team of scientists and engineers who would produce and deliver the desired technical products.  MITRE nominated Dr. Brant A. Cheikes to serve as the NRRC Program Manager for IKRIS.  Prof. Richard Fikes of Stanford University and Dr. Christopher Welty of IBM Corporation (who together conceived and proposed the original idea for IKRIS) were to serve as the IKRIS Technical Leads (TLs), and would be responsible for guiding the technical efforts of the IKRIS workshop team.

Beginning in December 2004 and continuing through early January 2005, MITRE worked with the TLs and ARDA to define a Statement of Work (SOW) for IKRIS.  ARDA identified two roles for MITRE: (1) to oversee the production of tangible deliverables from the IKRIS program, and (2) to facilitate technology transfer.  MITRE conveyed the revised and coordinated SOW to ARDA on 26 January 2005.  ARDA then released the funds and the IKRIS effort proceeded.  The overarching Project Work Statement (PWS) covering IKRIS was subsequently approved by the Government on 17 February 2005, allowing the official Period of Performance (POP) for IKRIS to run from 14 February 2005 through 1 October 2006 (nearly 20 months).  In mid-September 2006, a no-cost extension to 31 December 2006 was approved, to permit Prof. Fikes, Dr. Welty, and MITRE to prepare IKRIS deliverables and other reports for transfer to the DTO.

This is MITRE’s final report on its activities and accomplishments as NRRC Program Manager for IKRIS.  In this report we will only summarize the technical objectives and accomplishments of the IKRIS workshop team; details of the technical results will be provided in a separate report being prepared by Prof. Fikes and Dr. Welty.  This report will instead document MITRE’s supporting efforts.  In the remainder of this chapter, we document the workshop’s milestones (§1.1), participants (§1.2), accomplishments (§1.3) and impact (§1.4).  Chapter 2 describes MITRE’s support activities and accomplishments, and Chapter 3 presents conclusions, recommendations and lessons learned.

1.1 IKRIS Milestones (pg 1)

The IKRIS Workshop achieved several milestones during its 24-month lifespan:

• Project planning—December 2004 thru April 2005
• Kickoff meeting—25-28 April 2005
• Execution of technical program—May 2005 thru March 2006
• Community meeting—3-6 April 2006
• Capstone Demonstration—December 2005 thru September 2006
• Completion of technical work—30 September 2006
• Production of final products and reports—October 2006 thru December 2006

The IKRIS Challenge Workshop officially began at a face-to-face meeting held 25-28 April 2005 at the Columbia Hilton (Columbia MD). 
In attendance were 34 scientists from industry and academia (most of whom were coming under MITRE subcontract to execute the IKRIS technical program), plus 12 representatives from ARDA and the Intelligence Community. 
At this meeting, the basic organizational structure for IKRIS was established.
It was agreed that IKRIS technical work would be performed by five relatively autonomous Working Groups (WGs), each with a designated WG Leader and with a membership chosen by the TLs based on their assessment of each participant’s unique skills and interests.  

The WGs and WG Leaders were:

• Interoperability: Pat Hayes (Florida Institute for Human and Machine Cognition)
• Contexts: Selene Makarios (Stanford University)
• Scenarios: Jerry Hobbs (University of Southern California, Information Sciences Institute)
• Evaluation: Dave Thurman (Pacific Northwest National Lab)
• Technology Transfer: Paula Cowley (Pacific Northwest National Lab)

During the period May 2005 through March 2006, the five WGs conducted their activities independently, coordinating and collaborating using MITRE-furnished e-mail distribution lists, document-sharing services, and teleconferencing systems.
The IKRIS Executive Team—the NRRC PM for IKRIS and the IKRIS TLs—established a policy of meeting every two weeks (by teleconference) to review technical progress and discuss project finances and other management issues. 
Also on a biweekly schedule, MITRE hosted “All Leads” telecons (bringing together MITRE, the IKRIS TLs, and the Leads of each of the five WGs) to discuss technical activities, schedule, and related issues.
By March 2006, the Interoperability, Contexts, and Scenarios WGs had each completed drafts of their respective technical products. 
A second IKRIS community face-to-face meeting was held 3-6 April 2006 at the Computer History Museum in Mountain View, CA. 
The leader of each WG presented the group’s technical results in plenary, allowing peer review by the entire IKRIS community. 
The ensuing discussions motivated revisions and enhancements to the draft technical specifications, which were then refined over the period May thru September 2006.
In parallel with the core technical development efforts, a “Capstone Demonstration” effort took shape. 
Initiated by the joint efforts of the Evaluation and Technology Transfer WGs, and ultimately directed by Dr. Welty, the Capstone Demonstration sub-project put the fundamental IKRIS-developed interoperability approach to the test. 
Three analyst-support prototype systems—KANI, by the Stanford University, IBM Corporation, and Battelle/PNNL team, Nooscape, by the Cycorp team, and Slate, by the Rensselaer Polytechnic Institute (RPI) team — were selected from the suite of tools that had been developed under the auspices of ARDA’s NIMD (Novel Intelligence from Massive Data) program. 
Common to these three systems was their use of sophisticated knowledge representation and reasoning technologies to assist analysts with various aspects of intelligence reasoning and decision making.

The Capstone Demonstration team developed a realistic intelligence-analysis scenario based on a case study, called “The Sign of the Crescent”, obtained from Prof. Frank Hughes of the Defense Intelligence Agency’s (DIA) Joint Military Intelligence College (JMIC). 
The Capstone team then showed how the IKRIS interoperability solution enabled the three NIMD systems to work together, under guidance from an analyst, to formulate and test several hypotheses that are central to the DIA/JMIC case study.
The Capstone Demonstration activity was originally conceived in late 2005.  It took concrete form in February 2006 when the team was formally established and the broad outlines of the demonstration scenario were defined.  Significant progress was made during the April 2006 IKRIS community meeting, and the Capstone team continued to work with increasing intensity over the summer of 2006, coordinating their efforts with frequent teleconferences. 
A Capstone-specific face-to-face meeting was held on 13 September 2006, at the IBM facility in Hawthorne NY.  During this single focused workday, the Capstone demonstration reached a state of near-completion. 
The team continued to work through September to tie up loose ends and complete the project. 
The Technology Transfer WG delivered a briefing package documenting the demonstration storyboard and illustrating key examples of IKRIS-enabled interoperation among the three systems.
The IKRIS Challenge Workshop completed its technical work by 30 September 2006.  Wrap-up reporting work is expected to continue through December 2006.

1.2 IKRIS Community Membership pg 3

Over the IKRIS Workshop’s lifespan, the “IKRIS Community” grew to become larger than just those scientists and engineers directly funded to perform IKRIS research and development.  The complete list (excluding TLs and WG leaders) of all those who participated in the IKRIS technical effort (asterisks indicate those who participated without direct financial support) is below:

Bill Andersen (OntologyWorks, Inc.); *Fotis Barlos (BAE Systems); Danny Bobrow (PARC); Selmer Bringsjord (Rensselaer Polytechnic Institute); John Byrnes (FairIsaac/HNC Software); Alan Chappell (Battelle Memorial Institute/Pacific Northwest National Labs); Andrew Cowell (Battelle Memorial Institute/Pacific Northwest National Labs; Chris Deaton (Cycorp); Keith Goolsbey (Cycorp); Michael Gruninger (University of Toronto); *Ian Harrison (SRI International); Karl Heuer (Stanford University); Robert Hoffman (Institute for Human and Machine Cognition); Mario Inchiosa (NuTech); David Israel (SRI International); Charles Klein (Cycorp); *Hua Li (Sarnoff Labs); Arun Majumdar (VivoMind); David Martin (SRI International); Mark Maybury (MITRE); Drew McDermott (Yale University); Deborah McGuinness (Stanford University); Sheila McIlraith (University of Toronto); Chris Menzel (Texas A&M); Dan Moldovan (Language Computer Corporation); David Morley (SRI International); Leo Obrst (MITRE); Jennifer Ockerman (Johns Hopkins University/Applied Physics Lab); Valeria de Paiva (PARC); Richard Rohwer (FairIsaac/HNC Software); Andrew Shilliday (Rensselaer Polytechnic Institute); John Sowa (VivoMind); Joshua Taylor (Rensselaer Polytechnic Institute); *Marco Valtorta (University of South Carolina); *Russ Vane (General Dynamics Advanced Information Systems); Michael Witbrock (Cycorp); Wlodek Zadrozny (IBM Corporation)

It should be noted that most participants were provided with relatively modest amounts of funding for their efforts, mostly to cover travel expenses, but also to cover some technical labor delivery.  Some of these individuals were able to contribute time significantly in excess of what the IKRIS Workshop was able to fund, and we are grateful for their efforts.  
In addition, the IKRIS Executive team is grateful to our four “Government Champions”, who provided advice and technology-transfer guidance over the course of the project:  Steve Cook, John Donelan, Jean-Michel Pomarede, and John Walker.

1.3 Summary of IKRIS Accomplishments and Impact (pg 4)

These are the major accomplishments of the IKRIS Challenge Workshop:

• IKL—the IKRIS Knowledge Language. This is the key technical result of the IKRIS Challenge Workshop. The Interoperability WG developed a formally-specified language, based on an emerging ISO standard called Common Logic, into and out of which a variety of distinctly different knowledge representation (KR) formalisms can be translated. Using IKL as an interlingua, knowledge representation and reasoning (KR&R) systems are able to interchange knowledge, inference rules, partial proofs, etc., and thereby carry out fully or partially automated collaborative problem solving. In addition, IKL is a language that knowledge repositories may use for representing reusable knowledge in a way that is understandable by a broad range of KR&R systems.
• ICL—the IKRIS Context Logic. ICL is a logic formalism for representing and reasoning about context-dependent knowledge, including alternative hypotheses, points of view, world states and scenarios. IKRIS produced the ICL formalism, methods for translating knowledge represented in ICL into and out of IKL, and methods for performing effective automated reasoning with context-dependent information represented in ICL.
• ISIT—the IKRIS Scenarios Inter-Theory. The Scenarios Inter-Theory specifies an approach to translating among the principal formalisms in current use for declaratively representing processes (i.e., scenarios). ISIT consists of an ontology of terms for representing processes, and a set of “bridging axioms” for translating to and from other process representations. ISIT is represented in IKL and thereby enhances IKL’s capabilities as an interchange language by providing it with a representation vocabulary and translation methodology specifically designed for processes.
• Evaluation Report<b/>. The Evaluation WG has produced a report describing a collection of informal evaluations of the core IKRIS interoperability specifications. For example, by performing a series of “round-trip” knowledge-exchange experiments, each increasing in complexity and rigor, the Evaluation WG has shown that IKL is a sound and effective mechanism for knowledge interchange.
• <b>Capstone Demonstration. The Capstone Demonstration serves both as an evaluation of knowledge interchange using IKL, and as an illustration of the potential Intelligence Community impact of the IKRIS-developed approach to KR&R system interoperability. The Capstone Demonstration team produced (1) a detailed scenario in which an intelligence analyst develops and tests a sequence of hypotheses, (2) a storyboard presentation that describes how three distinct KR&R systems interoperate effectively using IKRIS-designed tools and techniques to assist an analyst as she develops and tests her hypotheses, and (3) a collection of knowledge bases (KBs) serving as a record of the execution of the Capstone storyboard by the three selected interoperating systems.
• IKL translators. IKRIS participants at Stanford University implemented a set of software tools for building automated translators into and out of IKL. They then used those tools to build example translators for use in the Capstone Demo. In addition, the RPI team developed translation software for their Slate system to read and write IKL knowledge bases.

In summary, IKRIS achieved or exceeded all of its planned outcomes.

1.4 Present and Future Impact of IKRIS (pg 6)

The Intelligence Community impacts of the IKRIS-developed technical specifications are significant, though as of this writing, largely awaiting realization.  First and foremost, IKRIS has developed a cross-cutting enabling technology that will facilitate wider use of KR&R technologies across the Intelligence Community.  Heretofore, specialized KR&R system “stovepipes” have been developed, often under IC funding, to demonstrate or deliver specific analysis-support capabilities to limited target user communities.  In many cases, these systems have possessed complementary KR&R capabilities, yet have been unable to leverage each other’s strengths due to their inability to “speak a common language.”  To enable “knowledge interoperability”, these systems need to be able to express their knowledge not only in a common syntax, but also do so in a way that preserves the intended semantics (meaning) of the expressed knowledge across system boundaries.  The IKRIS Challenge Workshop has addressed both aspects of this problem.

The IKL specification appears to fully meet the need for a common syntax, based on the set of alternative KR formalisms considered during the course of the IKRIS workshop.  That is, it is now possible to mechanically translate a well-formed expression in any of the target KR languages studied by the IKRIS workshop team to a well-formed IKL expression, and from there back to a well-formed expression in a target KR language.

The key feature of IKL is its ability to support the transfer of meaning across system boundaries.  As a result, we can now mechanically translate a knowledge base kbA of knowledge structures expressed in the native KR language of KR&R system A into a knowledge base kbIKL of knowledge structures expressed in IKL, and then from kbIKL into a kbB of knowledge structures expressed in the native KR language of system B, such that any sentence logically entailed by kbA  also is entailed by kbB.  This means that system B not only can incorporate knowledge transferred to it from system A, but also can perform automated reasoning using that knowledge. 

This ability to transfer meaning across system boundaries is essential because it enables automated collaborative problem-solving by KR&R systems having unique inference capabilities that are intimately tied to features of their internal KR languages.  In essence, IKRIS has set the stage for moving beyond the IC’s current goal of cross-boundary information sharing to the more challenging goal of automated cross-boundary knowledge sharing. 

Besides developing a specification for IKL that allows transfer of meaning among KR&R systems, the IKRIS workshop team took IC needs into account when designing IKL. Three IKL language features represent major technical accomplishments and are worth highlighting here: 1. IKL treats propositions and sentences as first-class objects in the language. This allows KR&R systems not only to express intelligence information as IKL propositions and sentences, but also to represent and reason about meta information, such as the security classification of intelligence information, its provenance, its credibility, its relations to other pieces of intelligence information, etc. 2. IKL supports the expression of relativized names. This feature makes it possible to reason effectively in situations in which one must distinguish between names and their denotations. For example, in an intelligence analysis scenario, we might want to represent and reason about multiple entities (persons, organizations, etc.) which might be known to different people by different names. So we might want to be able to represent the fact that the person whom John believes is called “Mary” is actually the same person whom Bill believes is called “Jenny”—or that a piece of weapons technology referred to by terrorist J as “the special shipment” is the same thing as “the new product” referred to by arms dealer B. 3. IKL supports the explicit definition of sortal restrictions on existence, and of relationships between sorts or types. This is because IKL content is often reliant upon some framework of classification of things into categories or classes, the primary use of which is to provide appropriate quantifier restrictions. Such a framework of categories is often referred to as a system of types or sorts, and many logics and notations are designed to conform to them, with special mechanisms for handling sortal reasoning or even allowing type checking to be done at parse time. IKL is not a typed logic in this sense, but it allows those restrictions and relationships to be made explicit for purposes of translating content into IKL from such a typed or sorted notation. Although the value of IKL is substantial, the additional value of both ICL and ISIT should not be overlooked. The ICL formalism provides a mathematically sound and rigorous foundation for representing and reasoning about alternatives, such as alternative hypotheses, interpretations of facts, chronologies of events, etc. This technology will allow the development of new analyst-support tools that aid hypothesis generation and testing. The Scenarios Inter-Theory shows how highly specialized knowledge about time, events, process inputs/outputs and preconditions, and cause and effect relationships can be transferred among systems in a meaning-preserving way. This is of particular value to the IC given that this kind of “scenario” knowledge is often fundamental to analytic reasoning and decision making. In the next section, we summarize MITRE’s efforts to facilitate the achievement of these important results.

2 Summary of MITRE Support Activities (pg 8) =

2.1 Administration and Logistics Support (pg 8)

2.2 Direct Technical Support (pg 9)

2.3 Summary (pg 12)

3 Conclusions, Recommendations and Lessons Learned (pg 13)

3.1 Conclusions (pg 13)

3.2 Recommendations (pg 14)

3.3 Lessons Learned (pg 15)