Our JourneyBy Kevin McGushion
InkaBinka 1.0
Our journey began on June 20, 2012 with the development of a technology we called an active webpage consolidator, which we patented. The basic idea behind InkaBinka 1.0 was to allow any user to follow many websites from a single screen. We let these users identify any portion of many websites and consolidate them on one InkaBinka homepage. InkaBinka then would continually update those portions so that a user could remain up-to-date on all their favorite sites from a single home page. My son Hunter and I laid out all of the specifications for version 1.0 and Chris Brahmer coded it.
My good friend, Woody Fraser, who has since passed, was a famous T.V. producer and he got me on Fox News to talk about InkaBinka. However, in order to get on the show, I had to first talk about some topic in the news that day. That topic turned out to be the Iranian rocket program. Given my Aerospace background, I agreed and before I knew it, I was on the Neil Cavuto Show. This interview is hard for me to watch now and I laugh every time I do because I am talking about "space monkeys". At the end of the space monkey interview, they let me talk about InkaBinka.
As the adage goes, if you are not embarrassed by your version 1.0, you launched too late. I'll just say, we didn't launch too late.
As the adage goes, if you are not embarrassed by your version 1.0, you launched too late. I'll just say, we didn't launch too late.
InkaBinka 2.0In the tech world, there is a term called "pivot". Pivot is what you do when you realize what you just made really wasn't everything it should have been. So Chris, Hunter and I pivoted to something I am passionate about: news and information. We called the first iteration of this new product "SMART", because we would make you smart on the news of the day in 3 minutes or less. SMART, which we later just called InkaBinka, summarized a news story in just 4 bullet points beside animated images of the story so that you could become knowledgeable about that story in just a few seconds.
InkaBinka 1.0 Composite Design April, 2013
InkaBinka 2.0 became very popular but it had a major bottle neck. Initially, I manually created all the summaries and later I hired someone to create the summaries. Needless to say, this would not work at scale and we were already being asked if we could summarize other types of content. Hewlett Packard, a vendor we were working with even wanted us to summarize company documents. We needed to invent something that could summarize news stories, documents, and even the entire internet automatically.
Our data center used new cutting-edge servers provided by Hewlett Packard called Moonshot servers. Because we were among the first companies doing new and exciting work with these servers, Chris and I were asked to attend an HP conference in Barcelona in December of 2014. At this conference we would talk about our implementation and success with their servers at InkaBinka. As luck would have it, after the conference, Lufthansa pilots went on strike and Chris and I were stranded. Worse things could happen as we were having a pretty good time in Barcelona, but being inventors, we were always in an unsettled state and had to invent.
Our data center used new cutting-edge servers provided by Hewlett Packard called Moonshot servers. Because we were among the first companies doing new and exciting work with these servers, Chris and I were asked to attend an HP conference in Barcelona in December of 2014. At this conference we would talk about our implementation and success with their servers at InkaBinka. As luck would have it, after the conference, Lufthansa pilots went on strike and Chris and I were stranded. Worse things could happen as we were having a pretty good time in Barcelona, but being inventors, we were always in an unsettled state and had to invent.
InkaBinka SMART January 2014
Wiring a Moonshot Server in our MDR Data Center
That chilly Sunday of December 7, 2014 in the empty restaurant of our hotel, I asked Chris, "What if we knew nothing? How would we make a technology that could automatically summarize news stories?" Without knowing it and without knowing the daunting journey that lay before us, we began to build an AI that would eventually become the foundation of all modern AI.
My approach to invention may be different from others but I intentionally do not look at the work others have done on a subject so as to not bias myself with a predetermined path which is not my own. In that restaurant, as we broke down a story about a Frenchman named Moreau who traveled to Syria to become a Jihadist, we began coding the likelihood of a next word in a sentence. We developed this method without yet knowing there was a technology called the Markov chain rule, we simply made a tool that calculated the probability of a next word given a seed first word or a series of words. We initially used random news stories to power our probability model but eventually found that using many stories about the same topic improved contest and logical outcome of our model. Without knowing it, we happened upon the notion of a small language model (SLM) or what was to later be called "expert mode".
By the time we got back to our offices in Camarillo, California we knew we had to build out our data center in Marina del Rey as well as increase our internet bandwidth. Based on the work in Barcelona, we knew that in order to summarize a single news story we had to scrape the internet for all stories written about that event during the same time period to build our SLM for that story.
My approach to invention may be different from others but I intentionally do not look at the work others have done on a subject so as to not bias myself with a predetermined path which is not my own. In that restaurant, as we broke down a story about a Frenchman named Moreau who traveled to Syria to become a Jihadist, we began coding the likelihood of a next word in a sentence. We developed this method without yet knowing there was a technology called the Markov chain rule, we simply made a tool that calculated the probability of a next word given a seed first word or a series of words. We initially used random news stories to power our probability model but eventually found that using many stories about the same topic improved contest and logical outcome of our model. Without knowing it, we happened upon the notion of a small language model (SLM) or what was to later be called "expert mode".
By the time we got back to our offices in Camarillo, California we knew we had to build out our data center in Marina del Rey as well as increase our internet bandwidth. Based on the work in Barcelona, we knew that in order to summarize a single news story we had to scrape the internet for all stories written about that event during the same time period to build our SLM for that story.
I photographed the Segrada Familia on December 5, 2014
We quickly learned that the Markov rule was inadequate in story telling because it would rapidly descend into nonsensical word strings. We had to make something that would tie words together based on context, so we invented the "Bridge Rule", something that would later be called self-attention. The bridge rule would link one or many words to others based on the contextual relationship of those words or the attention one word gives to another. We would track the ordinal position of these words so that we could know the order in which events and words occurred as well as the strength of their attention, this ordinal position tracking would later be called positional encoding.
Over the next two years we further developed the concept of self-attention and positional encoding so that we would know that, in the context of one particular story, the word "he" referred to "Barack Obama" or that the word "they" referred to the "United Nations". We created a query matrix that would be used to create a keyword matrix in order to determine the probability of a next word in a logical sentence, something that would later be called multi-head attention. We did all this extremely rapidly and bidirectionally with parallel processing which allowed us to ingest a huge amount of information and process it in milliseconds, especially with our SLM.
When we were finished, a person could ask our AI a question about the news of the day and it would answer in a manner that was indistinguishable from a human. More importantly, it would answer with words and expressions that no human had ever written before on that subject. We called this “abstraction” and we stood at the precipice of being able to pass what is famously known as the Turing Test.GoogleShortly after this achievement, an executive at Hewlett Packard, Paul Santeler, introduced us to Google’s M&A group and they arranged for a high-level overview of InkaBinka on September 17, 2015.
After that, we were invited to conduct a black box demo of our AI to Google on September 25, 2015. At that black box demo, one Google executive said of our ability to abstract, "I have over 100 NLP PhD's and AI researchers in the building next door and they can't do what you just did." Our invention was undeniably revolutionary to the most skilled experts in the field.
On October 12, 2015 I signed a preacquisition NDA with Google and they requested a technology disclosure which I gave. I assume, at a minimum, I was giving the disclosure to all 100 people in the AI building at Google since they could not abstract and we were the first people in the world who could. Shorthly after that Google made a low ball offer which I declined. In August of 2017 I shut down InkaBinka due to lack of funding.
Roughly 18 months after my disclosure, on June 12, 2017 Google authors released the now famous paper, Attention Is All You Need, where they described their “transformer” as being built on self-attention, query matrix, parallel processing and positional encoding.
Over the next two years we further developed the concept of self-attention and positional encoding so that we would know that, in the context of one particular story, the word "he" referred to "Barack Obama" or that the word "they" referred to the "United Nations". We created a query matrix that would be used to create a keyword matrix in order to determine the probability of a next word in a logical sentence, something that would later be called multi-head attention. We did all this extremely rapidly and bidirectionally with parallel processing which allowed us to ingest a huge amount of information and process it in milliseconds, especially with our SLM.
When we were finished, a person could ask our AI a question about the news of the day and it would answer in a manner that was indistinguishable from a human. More importantly, it would answer with words and expressions that no human had ever written before on that subject. We called this “abstraction” and we stood at the precipice of being able to pass what is famously known as the Turing Test.GoogleShortly after this achievement, an executive at Hewlett Packard, Paul Santeler, introduced us to Google’s M&A group and they arranged for a high-level overview of InkaBinka on September 17, 2015.
After that, we were invited to conduct a black box demo of our AI to Google on September 25, 2015. At that black box demo, one Google executive said of our ability to abstract, "I have over 100 NLP PhD's and AI researchers in the building next door and they can't do what you just did." Our invention was undeniably revolutionary to the most skilled experts in the field.
On October 12, 2015 I signed a preacquisition NDA with Google and they requested a technology disclosure which I gave. I assume, at a minimum, I was giving the disclosure to all 100 people in the AI building at Google since they could not abstract and we were the first people in the world who could. Shorthly after that Google made a low ball offer which I declined. In August of 2017 I shut down InkaBinka due to lack of funding.
Roughly 18 months after my disclosure, on June 12, 2017 Google authors released the now famous paper, Attention Is All You Need, where they described their “transformer” as being built on self-attention, query matrix, parallel processing and positional encoding.