Q: Why is the combination synergistic for research & applications?

A: There are uncountable applications for all of these areas:

a) improving machine vision/understanding
b) improving the common knowledge & representational base of VR to include more familiar human/man built environments & natural scenes
c) improving common sense reasoning about every day situations & possibilities
d) improving distributional knowledge about what is common/uncommon in different environments
e) sensing what holds audience interest
f) getting a sense of the interaction between language use & what is present or requested to happen in physical scenes.
g) improving smart video editing for meaning & focal interest and smart search.
h) evaluating how much scene context adds to speech understanding accuracy

Compared to machine vision in the field with LIDAR and/or stereo cameras & structure from motion, video interpretation is harder. But it is easily available to human adults, & we believe that competency plays a supervisory role in everyday vision. Similar remarks apply to the difficulty of understanding isolated speech recordings in a video.

A process can gain efficiency by working on all these things simultaneously in a computational setting. It’s a setting that fits a large research collective better than a small academic group. Combing multiple areas of study in the same work framework & pre-processing setup improves effiency.

Lookings at summary stats alongside good examples of “what went wrong” in a computational process helps researchers & engineers to focus efforts on overall quality improvement, with a justified sense of where algorithmic improvements are needed & justified.

For example of interest robotics – it’s relatively straight forward to built a catalog of videos with falls & crashes & then a feature similar set where that doesn’t happen, & then test whether machine prediction is at the level of human prediction.

The platform also provides a firm base for the drone surveillance work to add computational understanding from unfamiliar vantage points.