Most Important Optimizations in Software

[josh]

In futuristic developments, the most advanced interfaces for “understanding” whether formal domain models are both complete & correct will include NLP interfaces.

Practical NLP research always confronts a questions of both a) where is meaning anchored & b) how much knowledge must or should be included in the fragment of smart & common sense that our science/engineering development is claiming to supply or represent.

In other blog posts I try to suggest that a) can be linked to probability distributions over precise VR models of situations which can themselves be linked to objective/repeatable measurements/sensors, assuming fixed “experimental conditions”. Linking a) to behavioral correctness with respect to populations of language users is also relevant.

For b), we want to form an intersection of language used by professionals in the area & the subset of discourse that is actually understood by our interface as measured by its ability to behave correctly & make correct predictions about relationships to VR models & to other speakers novel discourse. This implies building bridges from both directions to fill that intersection with sufficient content & to be sufficiently “self aware” about what is understood & what is not understood by the engineering artifacts.

The ability to create test situations in VR, is it becomes sufficiently facile, will also give another route to testing model completeness & correctness.

The above points, taken together, suggest that an NLP interface for building VR domain/case models, linked to preexisting VR defiitions, could be a helpful tool for either “knowledge engineering” underlying domain specific NLP capabilities or for trained users testing & developing their highly specific case/domain models.

Example – Unreal interface

Case 1 Desc: Customer walks into a Credit Union?

Software: What kind of place is a Credit Union?

C1D: A kind of bank that is owned by & run for the benefit of classes of customers.

Software: What are the properties of the Credit Union that are not properties of the Bank, (or visa versa if you were speaking informally) that matter to the example?

C1D: Membership status of the customer might matter and other fetures of the lending available can be different…

Point is, it’s plausible that ‘bank’ is understood and ‘credit union’ is not. What would be the actual knowledge engineering or user interface to add the relevant details? It doesn’t seem like pictures of Credit Unions would help & encylopedias might be too disorganized for our level of auto-comprehension. We want to dynamically add to our VR catalog/distributional model the bits we care about. It’s development work. It’s not programming in Python/etc. We need to develop best IDE for doing that in order to follow this path successfully.

[josh]

Q: Fans of formal methods in software worry about vagueness & ambiguity in natural language & hope to escape from it with closed world definitions. Explain why you think that approach is wrong-headed in the ways that they care about?

A: A formal/mathematical system can be wrong in formal ways *or* wrong/irrelevant in terms of its correspondence to a set of reality situations that it wants to model/control/represent. Formalists put a lot of focus on the formal ways of being wrong: contradiction, incompleteness, non-executability, possibility of ifinite-loops or non-well foundedness is exeutable models, etc. These are real problems with models of any size. But getting them right doesn’t say anything about whether or not the problems of correspondence to reality have been solved. The formally healthy model can still be a complete disaster as a representation of the relevant problem.

In order to gain effort/cost efficiency, more participating eyeballs, more situated attention, more tests, & more correctly evaluated tests, we want the terminology of the formal model to be a good fit to the real world terminology & mental conceptions of the speakers & we want to reuse fine working chunks of domain modeling from other applications. This involves building & maitaining connections to natural language usage. We can say that *formal correctness* of the specification is still to be evaluated by correct performance on a specific set of cases. But achieving a large set of cases & evaluating them correctly and doing that at acceptable levels of cost will be far more workable when there are bridges to general & relavant domain natural language use.

[Author]

Posts