Natural Language Contextual Search (2012)

Tools: C++, speech recognition

This was an exploratory idea I had from my time at Intel RealSense group. I was advocating for these ideas within the context of the Dragon Assistant product, but it was deemed “out of scope” so I teamed up with colleagues within Intel to explore it.

Contextual Recall

​LifeTracker is a tool that helps users contextually recall information they encounter on the web or their personal computers by simply asking for it. At the core of LifeTracker is the principle that computers should be used as an extension of human capabilities. Although humans have a fascinating ability to manipulate information during reasoning, their capability to actually recall information is rather limited and often inconsistent. In comparison, from their inception, computers have been designed to have flawless memory. Furthermore, when trying to recall something, humans often remember the associated context, even if they don’t remember the exact thing we are looking for. So we designed and built a tool that locally indexes files, documents, media and visited web pages, parses the information for contextual information and using TFIDF and speech recognition allows you to contextually recall this information.

Prototype

A fully working prototype was developed in collaboration with the Software Services Group at Intel.