Leveraging Context-Awareness allows you to Self-Construct Ontologies

Well, there’s been a bit of a gap since my last post, but no gap in my thinking about what to post on!  Back in mid-June I hosted a panel discussion at the SemTech conference with colleagues and friends Jeff Jonas and Marc Davis.  Our (unstructured) panel was entitled “True Semantic Reasoning: Self-Constructing Ontologies.”  The discussion was fun, but in the feedback from the panel there was confusion on why we focused our discussion on context-awareness and not on the self-constructing ontologies (as implied in the title of the panel).

I’d like to explain!

The premise of our discussion was that Semantic reasoning in the absence of preexisting ontologies is done all the time, in fact has to be done in most cases today as ontologies simply don’t exist for most domains.  In the absence of formal ontologies, it is still possible to create semantic graphs, albeit with significantly less inherent structure than a formal ontology. In the case of our technology at Atigeo, the graph doesn’t express classes of relationship.  Without formal ontologies, reasoning can be achieved by using contextual information to help collapse the path in the semantic graph down to one that is less ambiguous or even completely unambiguous for the reasoning computation.

Think about how you reason to the name of the person I am describing;

· An Irish man

· Somewhat of a political activist

· Interest in 3rd World Charities

· Often seen wearing yellow sunglasses

· Is in a rock band

· Is friends with The Edge

· U2

Well, the person in question is Bono.  Of course in the presence of a music ontology we could easily perform the reasoning needed, although we may find some of the pieces of information in the description above are tertiary to a music ontology and as such not understood in the semantic reasoning process. "Yellow sunglasses," for example, tends to trigger recognition with many people I try this example with.

The type of self-constructing ontologies we have been working on leverage large volumes of unstructured data related to a domain, in order to discover association-based relationships between the concepts expressed in that domain.  The approach generally induces a large and widely connected graph associating concepts which can be queried by defining a list of concepts that contextualize the query. In the case of the example above, the list of concepts would include: Irish, political activist, 3rd World Charities, yellow sunglasses, rock band, The Edge, and U2.  This contextualization allows us to collapse the modeled semantic ambiguity in the problem (the complex graph on the left in the image below) into a much simpler graph on which inference can be performed (on the right, below);

Another compelling example can be built around Apple’s new 4G iPhone. Self-constructing ontologies could be leveraged to assist customer support specialists in determining when an incoming complaint is associated with the “death grip” issue – one that has a lot of ambiguity in how users will describe it. In this case a list of concepts that could contextualize the query would be: antenna, dropped call, hand position, and phone case. It would be labor intensive to keep a traditional ontology up to date with the latest breaking information regarding this issue.

On our SemTech panel, Jeff Jonas spoke similarly about how increasing contextualization allows technologies to resolve ambiguity in identity or to connect multiple, apparently distinct identities.  This is akin to discovering new content expressing a new semantic relationship in the graph, allowing previously isolated sub-graphs to be connected.  Jeff writes very eloquently about this in his blog post “Puzzling: How Observations Are Accumulated Into Context.” Associating “death grip” and “dropped calls” is a great example of this. Persisting the context becomes the persistence of the ontology expressing current knowledge of the problem domain.

Mark Davis illustrated this same concept in the space of location information, with the notion that location-based context implies a certain semantic conceptualization (as illustrated by Yahoo! TagMaps).  Essentially, tagged and location stamped images uploaded to Flickr generate semantic associations with locations, which embed the concept of location into an ontology.

So the topic of our panel was relevant because self-constructing ontologies must leverage context awareness in order to discover association-based relationships between the concepts in a particular domain. Additionally, real-time maintenance of such ontologies relies heavily on contextual relevance because they do not have inherent structure.

As the demand for mobile access to information and services continues to increase, true contextual awareness will become a more pervasive requirement for ontology utility (scalability and real-time capacity). According to Gartner,

"By 2015 context will be as influential to mobile consumer services and relationships as search engines are to the Web. Whereas search provides the ‘key’ to organizing information and services for the Web, context will provide the ‘key’ to delivering hyperpersonalized experiences across smartphones and any session or experience an end user has with information technology. Search centered on creating content that drew attention and could be analyzed. Context will center on observing patterns, particularly location, presence and social interactions. Furthermore, whereas search was based on a ‘pull’ of information from the Web, context-enriched services will, in many cases, prepopulate or push information to users."

Posted in Uncategorized | Leave a comment

Algorithms for the right time web?

The experience David describes in "The Right-Time Web" broadly fits in the "context -aware computing" arena – the intersection of where you have been, where you are, where you will be and when, and the topical/semantic context around each spatio-temporal location and your activity there and the intersection of those things with others both socially and professionally as David describes in his examples.

Historically, context-aware computing has been limited by our ability to have sensing and computing with the user at all times. Today we are way beyond that, with our Smartphone devces bristling with sensor technology and plenty of computing power.

What is missing is a framework in which to establish measurements of relevance. I don’t mean your grandpa’s Information Retrieval-based view of search result relevance, I mean a measure of "this gets me/us and makes sense, right now". At the heart of enabling this, I believe, will be systems that can jointly reason with both physical and semantic attributes, dates, times and coordinates, in addition to the pertinent topic-based semantics – watch this space!

Posted in Uncategorized | Leave a comment

The Right-Time Web

(By my friend and colleague David Boardman @dbboardman)

A new buzzword, “right-time web,” has generated a great deal of interest recently. Effectively, it illustrates how the exponential growth of the “real-time,” web where live streams are continuously fed to us via social networks and media properties are of little use unless they are accessible at the right-time – in context. Supersaturated with real-time information, we don’t have the capacity to make immediate use of the endless amount of content that is generated. In looking at my Twitter feed this morning, I probably don’t care where my old college buddy had dinner and drinks last night in Austin, but when I travel there for business next month – that info suddenly becomes very relevant: right-time.

At Twitter’s Chirp conference April 15, Venrock’s David Pakman said, “The ‘right-time’ Web is more valuable in some cases than the real-time Web. Real-time data is only interesting when I’m actually looking for that information. There’s no service today that’s giving information when it’s really needed. If your company is doing that…I brought my checkbook.”

Pakman stated, “What is really needed is a service to collect, organize, and make available all the data shared by my networks.  Some call this social search.  I call it something bigger.  Life search.”

“Life search” is an interesting take on where the web is headed. After all, the most important piece to realizing the “right-time” web is YOU.  For systems to know when it is the right-time they need to have a deep understanding of you in context. This understanding of you will be derived from your personal data locker, which will include, or bring together, preferences, data, possessions, social graph, location data, your roles, and your intent. In addition to the personal data locker, the “right-time” web requires semantic understanding of content and websites and applications that leverage personal data lockers to present “right-time” interactions. Agents like Siri are in the early stages of providing similar services.

A look into the future: The “right-time web” in action

My personal data locker contains, among many other data points, my affinity for U2 and live music, 3 previous U2 premium ticket purchases, I live in Seattle, I purchased a ticket to LA for a business trip June 7th, and my friends Joe, Will, and Neil like U2.

I go to my favorite right -time web portal, application, or agent and express that “I want to go a U2 concert.” The agent responds with three options:

1) You will be in LA on business June 7th and U2 is playing in Anaheim.  Your client at XYZ Co. has been tweeting about the U2 tour.

2) U2 is playing on June 20th in Seattle and your friend Joe may be interested in going with you.

3) U2 is playing in Oakland on June 16th and your college friends Neil (lives in SF) and Will (lives in Dallas, but has a trip to SF registered in Trip It for this date) may be interested in going and Alaska Airlines is running special fares to SFO.

Option 3 sounds the most exciting and so my right-time web service provides me all of the necessary travel arrangements to choose from. I am presented with flight deals for Alaska Airlines (my preferred carrier) and Virgin America (at a slightly lower cost). I see hotel options from the Marriott (preferred) and at the Hilton, which is one block from the concert and Will is premium club member. I check out relevant aggregated feeds and reviews from trip advisor, tweets, four square check-ins, and gowalla pictures. I checkout premium tickets available at TicketsNow.com leveraging buyer reviews from social graph regarding sellers. I invite Will and Neil to collaborate at a convenient time (proposed by my agent based on inferred availability) OR a threaded conversation over email, facebook, twitter, or other appropriate messaging services.

The service sets up a Skype session at 8:30 pm – Will, Neil, and I accept the invitation. We jump on Skype, see relevant data (flights, hotels, reviews, tweets from each of the social graphs) in our right-time widget, and decide to stay at the Hilton because Will gets free upgrades to a suite and his company will be paying for it. The agent offers to adjust his booked flight on American, and I confirm my flight on Alaska.  We decide to go with the premium seats offered through a secondary ticket service.

Imagine throughout the weekend the “right-time” web helps the three of us find the right places to eat and bars to visit.  Right-time information is presented based on our location, college buddy roles, shared preferences (likes and dislikes) AND influenced by relevant real time data (facebook, twitter, foursquare, gowalla, yelp) available from our social groups. The Right-time web allows us to leverage the value of the real time web that we otherwise would have missed out on.  All of the real time data that flies by at the wrong time is now brought into focus and improves our experience. 

What does it mean for business?

The building blocks for the “right-time” web (personal data lockers, linked data, semantic understanding, and agents) will provide businesses the ability to offer customers more relevant, convenient, and useful content and experiences. 1:1 marketing is not a new concept, but having the ability to actually do it at the “right-time” will be.

Hurry up world.  I want my “right-time” web!

Posted in Uncategorized | Leave a comment

I Want My Personal Data Locker!

(co-written with David Boardman http://www.twitter.com/dbboardman)

David Boardman

The personal data locker.

Do you remember getting your first locker? I do. It was 7th grade the first year of middle school. The big leap from elementary school into the land of giants – 7th, 8th, and 9th graders.

On the first day of school we all anxiously awaited as our home room teacher handed out assignments. We all leapt from our chairs with locks in hand and went out to find the space that we would personalize and call home for our books, snacks, gadgets – and more. In a world where our parents – and school authorities – had visibility into and control of all of our storage spaces (well ok, not all – there was that tree fort in the woods) – this was the one place we could keep things private and retrieve them when we wanted. I remember walking the halls looking at all of the big kids lockers and how quickly they made them their own. Some had pictures of Cal Ripken, summer trips with friends, and u2 ticket stubs. Too cool.

Today there is a lot of talk about “personal data lockers,” as described in David Siegel’s book Pull. Lockers that will hold our music, our photos, our medical records, a record of our purchases, places we have been, people we know, and more. Personal ontology will describe all of the items in the locker so that applications will know how to access, manage, and act on the items in the locker.

Too cool. I am as anxious now as I was as a seventh grader to get this new locker, so I can start loading it and personalizing it. I can’t wait to load the entries for all of my purchased videos and audio, store all of my identities and preferences, my digital tickets – and ticket stubs. I want my TV, camera, iPhone apps, car navigation system, and web sites I use – to access my locker.

I want to configure who gets a key to my locker and what items they get to access or act on under specific contexts. For example, I want Hertz and Marriott to get access to my identity, location, and preferences when I am planning a trip, but not when I am going to the park with my kids. I want my Facebook friends to be able to get my music preferences so they can invite me to concerts I may be interested in. So much promise. So many questions.

Will I be able to do all of this? What will it mean for my life – better services, greater productivity, and cheaper goods? Who is going to give me the data locker – a bank, a mobile operator, Facebook, a new start-up? Will I have one locker for everything or a series of linked lockers containing different data for different uses? Will I be able to buy insurance on my locker? Will I be able to describe what happens to my locker in my will?

I am more excited to get the key to my new locker than I was as a 7th grader. Hurry up tech world. I want my personal data locker!

Olly Downs:

The personal data locker David described above requires a rich understanding of the relationships between different themes and concepts that can in part be learned from the actions that people take.

However, equally important is the learning derived from how things are described and talked about relative to one another (which really conveys what something means). With the use of hierarchy-free ontologies, Atigeo’s xPatterns enables systems, in real time, to determine semantic relationships between concepts – simply from reading and reviewing large bodies of unstructured text information about the domain.

In Atigeo’s version of the Personal Data Locker, the contents are expressed in unstructured form as lists of lists of concepts in combination with semantically-expressed spatial, temporal, and behavioral context. When this data is applied to xPatterns indexes of content semantically appropriate actions, items, and offers can be presented to the user at a meaningful time by the content provider.

The technology allows a unique approach to privacy; to assess a particular piece of content, the attributes within the personal data locker do not need to be shared with the content provider. The content description and any semantic metadata are simply indexed with xPatterns and under the permission and control of the owner of the data locker, Only a “score” of the content affinity to the data locker owner is presented to enabled content providers.

Posted in Uncategorized | 3 Comments

Getting Boxed into the Personalization Corner (or Olly’s Problem with Pandora)

You can tell someone lots of facts about you, but what helps them “know” you is how they understand those facts.  Take for example my musical tastes – I can tell you I like “The Crystal Method,” “The Shamen,” “Simple Minds,” and knowing all about music you could certainly pick some things I like.  But for me at least, there’s a problem, which I was reminded of a few weeks ago, when reading Mike Melanson’s ReadWriteWeb article about Google’s Custom Search

Mike’s metaphor, “Customizing search results, it would seem, can be like putting us in an echo chamber of similar ideas and opinions…we’re suddenly being subtly driven back to our own world view, as repeated by our peers,” rings true for me and and highlights a notion that I experience with many 1-to-1 personalization solutions deployed today; you can tell them lots of facts, but they struggle to “know” you.

I have broad musical tastes characterized explicitly by some of the artists I mentioned above; however, Pandora, despite its wealth of expertise in music genomics is not equipped to comprehend that my preferences are deeply entrenched in a broader personal context. If I were to tell you that I grew up in the UK in the ‘80s and ‘90s, have played jazz and classical violin since the age of 3 years old, I bet you could pick me a much broader and more satisfying playlist than Pandora with the artist information above.  And let me tell you, it wouldn’t be hard –  I’ve pretty much given up on Pandora, because I find I like so little of the directly related musical content it recommends that I tend to box it into a corner with all my “thumbs down-ing.”

Google’s customization effort is trying to leverage a broader view of you as a user, but as a user you can only passively engage it – and leveraging the music preferences of my Facebook social graph would certainly help expand Pandora’s actionable information. That said neither can capture, express, and act on the more personal statements I made above.

Posted in Uncategorized | Leave a comment

Breaking the Shackles of the Schema with a More Humanistic Query Language

(co-written with David Boardman http://www.twitter.com/dbboardman)

David Boardman:

Many years ago I was on a trip with my wife across Ireland.  For those of you who have made the trip, it’s highly likely that you have a piece of Waterford crystal sitting prominently in your home.  My wife and I wanted something a little different.  Over the course of 3 weeks, we scoured the shelves of many crystal glass shops in little towns throughout the countryside hoping to find a uniquely personal memento from our adventure.  Stay with me…this is going somewhere.  At about the 12th shop, while my wife was looking at crystal vases, my eyes wondered to a bulletin board behind the shop owner where a U2 ticket was tacked with great pride.  Now that caught my interest!  I leaned forward to read the details of the ticket: Slane Castle on September 1. 

Wow – a chance to talk to someone who was there. I piped up, "What was it like seeing U2 at Slane Castle?"

"It was crap," he abruptly replied. 

What?  I was stunned.  Why would he have the ticket so proudly displayed behind the counter? 

“I don’t like U2.  They sold out Irish people and our culture.  I went to see Coldplay.” 

I took a second glance at the ticket, but did not recognize the band in fine print – Coldplay.

Instead of creating casual conversation to align and resonate – I was completely stunned and shutdown.  I returned to looking at the hundreds of beautiful pieces of crystal displayed in the shop. 

We ended up buying a small vase and it’s proudly displayed on my piano at home.  Every time I look at the vase I remember my surprising interaction over the U2 ticket.

Fast forward.  It’s 2009 and I’m working with Olly to develop a way for technology to connect people quickly and meaningfully with the environment around them.  I was struggling to find a way to expose a profile to third parties that could query user preferences and enable meaningful action.  In the previous blog entry, we shared the challenges of creating an actionable unified view of the customer.  How do you get parties to agree on what data to collect, what format it should be stored in, and the allowed values? 

So what does it take to build a more humanistic query language?  To ask questions like a human?  Well – let’s take the example from my trip to Ireland.  As humans we leverage our understanding of the individual, the context, and the domain – to make decisions.

The chap standing behind the counter in Ireland had a persona – including preferences and more.  I had very little insight into his persona. I was operating in a domain – North American music.  I was operating in a context – a shop with the ticket on the board. I performed a mental query and the result was, "talk to this guy about U2".  Ouch.  Unexpected result!

With additional understanding of his persona, a greater understanding of the context of the concert (the buzz around Coldplay), and a better understanding of the domain (undercurrent of feelings towards U2 in Ireland or Coldplay –   I would have asked him about Coldplay instead of U2 and we would have hit it off.

Olly Downs:

At Atigeo we have developed an approach which has strong analogy to the human reasoning that David describes as a solution above. The declared and observed profiles in our xPatterns product provide a rich representation of the user persona (or Personal Data Locker as described by David Siegel in his book “Pull”).  We create current encyclopedic understanding of a given domain, represented in the domain expert, (see previous post on hierarchy free ontologies), that understands the similes drawn by both critics and fans between U2 and Coldplay, and Coldplay’s one-night stand supporting U2 at Slane Castle.  xPatterns indexes and understands data in the same way that David understood the U2 ticket via a query language that allows xPatterns to reason with context and the user persona in a specific domain to retrieve data.

So pretend for a moment that David and the store owner were computer programs.  Today, David and the store owner would each represent their music knowledge, tastes, and experiences in a schema or ontology.  A ticketing company would describe their ticket in a fixed format. Sensors in the environment would model context in a well known standard schema, like MPEG 7.  Now – assuming that the David application, the store owner application, the sensor manufacturer, and the ticketing company all bought and deployed the same ontology or schema (highly unlikely) then queries could be performed using a standardized query language such as SQL or SPARQL.

The challenge for Semantic Platforms isn’t the existence of the tools – the semantic web offers the toolkit, but as has happened historically with other attempts to standardize data interoperation (EDI, CORBA) the challenge is the lack of pre-constructed ontologies and semantic data dictionaries according to the standards, against which the tools can be leveraged.

Our approach offers a much needed alternative.  Will it solve every problem with exact precision?  No.  Will it create new solutions to existing problems that could not be cost effectively solved by machines in the past?  Absolutely.

Posted in Uncategorized | 1 Comment

Unified View of the Customer. Let’s Do It!

(co-written with David Boardman, twitter.com/dbboardman)

A battle cry heard around the world in enterprises and start-ups.  Hundreds, if not thousands of these initiatives start every day.  From the entrepreneurial social networking start-up to the world’s largest enterprises.  They all start by getting a group of people to decide on what data to put on the profile.  Sounds easy right? Try it. Get any group of more than 3 people, much less departments within an enterprise or competing corporations, to agree on what data is important to collect.  Next, hand out a schema to 3 more people or enterprise software providers, and convince them to change their system to query your definition of the consumer.  It’s hard.

There is hope.  The semantic web promises to make all of this easier.  The semantic web creates a world wide web in which the meaning (semantics) of information and services on the web is defined, making it possible for the web to "understand" and satisfy the requests of people and machines to use the web content.  The fuel of the semantic web is ontologies.  It’s possible to envision a semantic web where ontologies and tools are used to develop a deep understanding of a customer with a standard set of tools to access and manipulate. 

Sounds great.  But the semantic web isn’t here yet.  Do we need to wait until the ontology and schema are standardized by the industry? 

We have a proposal to change everything, where we realize the vision of the semantic web now, while we wait for ontologies to emerge and gain mass adoption.  We call it Schema-Agnostic Action on a Profile. 

Imagine if you could launch a unified view of the customer initiative and didn’t have to get a group of people or departments to agree on what data to collect or what format that data should adhere to.  Now imagine third party applications could query this profile without understanding the structure and content.  We assert that this transformational approach would allow more unified view of the customer initiatives to take less time, require less effort, and yield greater results. 

Schema-Agnostic Action on a Profile

We want to share what we think is one of the most differentiating concepts behind the technology we have been working on recently at Atigeo – the notion that it is possible to act on the attributes of an entity without knowledge of the schema with which that entity is represented.

To share or cooperatively act on the profile of an entity (business, person, object, thing) today requires agreement between the parties interacting in terms of the profile or content attributes/metadata, take for example MPEG7/21 efforts around media metadata.

Most commonly, systems interact through query-style interfaces where results are returned based on attributes or filters on attributes that are matched.  Among the most sophisticated systems today, some are able to step beyond attribute matching to exploit ontologies, graphically-expressed relationships between attributes or attribute sets.  These allow relationships that are understood (“my sister is my son’s aunt”, “the Nexus One is a Google Android-based cellular phone).

Not only do the underlying semantics of some domains evolve rapidly, but often "local terminologies" develop for the same concepts that subsequently need to be reconciled. (BTW – the latter turns out to be a very interesting graph-theoretic problem – see the work of the SHER team at IBM Research).

There are many domains for which there isn’t a standard data format. Additionally, there are many more domains for which, until the Semantic Web becomes pervasive, ontologies don’t yet exist.

Our technology is a solution to automatic hierarchy-free ontology discovery. As a corollary, this allows us the ability to determine affinity of an unstructured profile to content in the absence of “structured/direct match”. Uniquely, the ontology discovery process allows learning, refinement and expansion through user interaction, and through real-time tracking of content generated in the domain of question.

Ok – Got It.  But So What?

Imagine an enterprise launches yet another unified view of the customer initiative.  Now instead of teams of people battling it out over what attribute should go on a profile, we just set up a profile and start attaching data to it.  Interesting.  Now imagine handing out a set of APIs to enterprise IT staff and 3rd party application developers that allow them to act on the unified view of the consumer without providing a schema in a domain for which there isn’t an ontology.  Possible?  With a more humanistic query language that understands the meaning of the data, the process, and the persona, applications can be built quickly without being constrained by the definition of the data. 

In our next blog post we will explore the proposed framework for a transformational query language that breaks the schema and ontology shackles off of the developer and the data they are acting on. Stay tuned.

Posted in Uncategorized | Leave a comment