Digital Reasoning Systems is looking for Java expertise.

Please contact Harry Schultz to learn more about our opportunities.

Today we announced that our complete product line and services are now available on the GSA Schedule number GS-35F-4153D.

Read the full GSA schedule press release here. Special thanks to Intelligent Decisions, a solid partner and VAR reseller for us.

Net effect: It’s easier than ever for government agencies to take advantage of Digital Reasoning solutions. Please contact us if you have any questions or would like to learn more about procuring our solutions via the GSA schedule.

From the press release:

Digital Reasoning Systems, Inc.,the intelligence-software innovator, today announced that its product line is now available to federal agencies on the NASA Solutions for Enterprise-Wide Procurement (SEWP) contract. A SEWP listing allows all government agencies the ability to procure the company’s products at discounted prices.

Digital Reasoning products added to the SEWP contract are:

• Interceptor: Interceptor allows you the ability to look through all of your data rapidly and easily discover what is inside
• GeoLocator: GeoLocator is a tool that extracts populated places from your data and returns the extracted locations with their geo-coordinates
• Synthesys: Synthesys allows you the ability to easily create applications that leverage vast amounts of unstructured data

The stated vision of the SEWP contract is “to be the premier customer-focused contract vehicle for Federal Government purchase of mission critical, state of the art IT products”.

“Its a major coup for Digital Reasoning to be added to the SEWP contract. It makes our unique technology accessible across the Federal Government and positions us as a true platform for unstructured data analytics”, said Tim Estes, CEO of Digital Reasoning Systems.

Read the complete press release here.

From the press release:

Digital Reasoning Systems, Inc.,the intelligence-software innovator, today announced that Intelligent Decisions is now a Value-Added Reseller for Digital Reasoning’s full line of products and services.

For over 20 years, Intelligent Decisions has provided some of the most sophisticated and innovative solutions for state and local government, commercial organizations and almost every government agency, including the Intelligence Community and even the White House.

“In the last few years, Intelligence and Security initiatives have done a great job of collecting data, but they are now faced with having too much information. We see Digital Reasoning as a leading tool for understanding unstructured data automatically. This cutting edge technology can optimize several levels of analysis, allowing analysts to concentrate on the bigger picture”, said Roy Stephan, Director of IT Architecture and Engineering with Intelligent Decisions, Inc.

“We here at Digital Reasoning are proud to be selected by Intelligent Decisions as their first partner in the area of unstructured data analytics. They’ve been at the forefront of bringing several novel commercial technologies to the U.S. federal government and we believe we will join those leaders as our technology is adopted more broadly in the federal space through partners like Intelligent Decisions”, said Tim Estes, CEO of Digital Reasoning Systems, Inc.

Read the full press release here.

From the press release:

In a recent article for CNET News, Stephanie Olsen explained that investment in web technology initially dealt with commercializing the Web, helping companies like Amazon.com and eBay get on their way. The second wave of investment has been about helping people socialize and connect through sites like Flickr, YouTube, and Facebook. The third, she writes “will be about making sense of all the data people create around the Web, and then searching for patterns in the data to improve the delivery of personalized content, search results, or advertising.”

To make sense of the data, Olsen proposes, will require “building an intelligent system that understands the relationships between Web sites and how people use them–with the use of algorithms that understand keywords, context, and natural language on a massive scale. VCs (Venture Capitalists), for example, are looking to so-called semantic technology to significantly boost the amount of searches that result in an advertising “click.” Right now, an estimated 30 percent to 40 percent of Web searches do not return advertising revenue. But if a search engine understood the context of a person’s Web search more often, those numbers would improve, they say.”

“Simply put, the problem is information overload - there is so much good information that you have to look really hard to find the great information that you care about most”, said Tim Estes, CEO of Digital Reasoning Systems.

The typical approach to understanding unstructured data involves having to either read the data manually or use a keyword search tool. Both methods present challenges to accuracy and efficiency. Manual reading, while reliable, can take an inordinate amount of time. A keyword search, while fast, returns only limited results.

At Digital Reasoning, we apply advanced algorithms to solve both problems without sacrificing quality. In fact, because the software builds its models of meaning from the data and understands concepts - the end product is better information.

Since 2002, Digital Reasoning has worked with a variety of organizations and agencies both large and small to help them make sense of what is in their data. We are proud of the patented technology we’ve developed and our clients’ successes in the federal and intelligence market. Now we are making that same technology available to commercial clients.

Read the entire press release here.

You may have noticed SynthesysSDK.com.

This begets the obvious question: What is the Synthesys Platform?

The short answer: Digital Reasoning ’s Synthesys Platform provides the first true Software Development Kit (SDK) and server platform for Unstructured Data Analytics (UDA).

The slightly longer answer: The Synthesys Platform helps you find unexpected, critical knowledge hidden in your data. Synthesys takes unstructured text as input, uses entity extraction with strong semantic relationship analysis to operate on the input, and then outputs abstracted knowledge objects. You can then use these objects (people, places, connections, etc.) to understand and analyze what’s important.

For an in depth answer and to speak to us about possibly joining our limited beta, please contact us via the form on SynthesysSDK.com.

You can also attend one of our upcoming events or tech talks, for example my “Hacking the Meaning in Human Communication” presentation at the upcoming Tulsa TechFest in early October.

Watch for much more information on the Synthesys Platform on this blog in the weeks to come.

There are two possible outcomes:

if the result confirms the hypothesis, then you’ve made a measurement.

If the result is contrary to the hypothesis, then you’ve made a discovery.

Enrico Fermi

A couple of years ago we started a process of programming that was very different than anything I’ve seen in the last 15 years or so that I’ve been at it. We had a challenge given to us to produce a geographical location service built upon our entity extraction technology. It was an interesting exercise which at the time we had no experience doing. The object of the game is to read in text documents, discover location references, disambiguate them, look them up in a gazetteer and mark them up with the coordinates. This can be done either as an additional final section or, the more difficult case, in-line.

So off we went. Now the very first attempts at measuring this were done by me. I had had a lot of statistics in college but never thought I’d really get to use it. I came up with my own measures which were pretty close to recall and precision. Giving both numbers just didn’t fly with the management at the time. It was confusing. They wanted one number. After a little research I discovered both recall, precision and the mysterious F1 (or F-Measure).

 In the case of this task we defined tokens as either relevant or irrelevant. If the token represented a PPL (populated place) then it was relevent. Otherwise it was irrelevent. So if a relevent item was marked up with the correct location it was a true positive. If it was not marked up or marked up with the wrong location it was a false positive. If an irrelevent item was marked up it was a false negative. The debates raged on what to do in the case where the system found a location but just did not disambiguate it correctly and over what to do when tokens were improperly co-located (as in what if “Rio de Janeiro” came up as “de Janeiro” instead.) Ultimately we decided to keep it simple. Any error below the level marking something right or wrong was deemed just a detail.

It took a lot of measurements and a lot of debate but we got it to work. This learning process produced a lot of healthy discussion and when we did finally decide on what formulas were best everyone could clearly see how to proceed.

The first day we calculated the f-measure of our geo-coordinate markup service it came up an astoundingly low 37 out of 100. I went over the numbers several times. Management wasn’t happy. What was decided next ended up being a great model for future development. We were put in a conference room with our computers and a white board. We were told not to leave until the f-measure was above 80. The way the development worked we had one person who did work on the trained categories system and another guy who did the application programming. I was doing measurements and creating reference sets. Three of us working towards one task, side by side.

We would discuss potential strategies and would then run them through the test harness. Every strategy would impact recall and precision. Often this would show how these concepts are opposed. As one is increased the other is decreased. What you are looking for is opposition that is not equal such that the f-measure rises. You want the decrease to be smaller than the increase. While it seems obvious most people don’t program that way. They come up with a bunch of ideas, implement them and just accept the measurements they get. In our case each change was tested. Yes it was slow but it separated out the good ideas from the bad ideas. We also, in this way, discovered other weaknesses that were fixed. If we had not been looking at this on a case by case basis we would have missed the subtle clues that helped us iron out the other parts of the system that were contributing to the final result.

I believe that honestly measuring your tools’ accuracy is important not just for sales and customer reassurance but also for the whole development life cycle. Efforts are underway to allow the unsupervised portion of the DRS system to aid in getting the Geo Reasoning system at or above 90 f-measure. Right now 75-80 is state of the art. Every point of f-measure gain beyond 80 is far more difficult to achieve than all the ones prior. However a learning system should be capable of this feat. More on that later.

Matthew Russell, Director of Advanced Technology, joined Digital Reasoning in October of 2007 and has been making an impact since day one. A talented and dedicated programmer, Matthew’s long hours and creative energy have been focused on improving the Interceptor user interface, architecting web applications and devising innovative ways to embed the company’s core technology platform into commercial products . It has been a busy year for Matthew. Despite the relocation to Nashville and devoting countless hours to the work at Digital Reasoning, Matthew managed to complete his first book -”Dojo - The Definitive Guide” - published by O’Reilly Media and released on June 17th. We talked to Matthew about his book, the writing experience and plans for the future.

Q: What is Dojo?
A: Dojo is a piece of client-side technology - Javascript based - that creates great user experience on the web. It’s a toolkit, technically speaking, it’s something you can use to create a great user experience in a web browser.

Q: What makes Dojo superior to other Javascript toolkits?
A: The overall architecture is very well thought out. It’s industrial strength, it’s battle tested. Big blue chip companies are using it. And it has tremendous breadth and depth. It doesn’t just solve a little narrow problem, it can solve lots and lots of different kinds of problems, but the solutions aren’t just cursory…they are very involved.

Q: How and when were you introduced to Dojo?
A: At a previous company, a colleague and I worked on all these applications for the intelligence community and one really common issue with intelligence datasets was that there was generally a lot of data that needed to be displayed in a tabular format. We started to scope out what other people have done…other technologies in the Javascript toolkit realm and Dojo was on of those. From there, I started to learn all the other things Dojo automates and makes simpler.

Q: What other writing have you done and how did this book come about?
A: I had a great professor while studying Computer Science at the Air Force Academy who was my thesis advisor and he cultivated writing in a way that while writing your thesis you would produce enough materials for white papers and technical papers. I started writing fairly frequently for O’Reilly on the MacDevCenter site at the time. So, I had been doing development for Dojo and thought it would be a neat thing to write about. I sent in a pitch for an article on the topic and it sort of escalated and eventually someone got back to me and said maybe we want to write something bigger…maybe a book.

Q: How long did it take to write the book and what was that experience like?
A: The actual book writing process took roughly 10 months. I signed contract last July and I put the finishing touches on it the first week of June. I would estimate I spent roughly 1200 hours writing the book. One thing about writing a book - it’s not just about knowing the material from a technical standpoint. There’s so much overhead. How do I organize these thoughts? What information do I put in what chapter? What’s the most logical ordering for chapters? How do I keep the content written in such a way that it engages the reader and doesn’t become boring, dry, technical material? I think I stayed true to that O’Reilly style of keeping it fun and engaging the readers. The hardest thing about writing the book in my opinion is that it has always been a moonlighting effort for me, it’s not my daytime job. So, if you can imagine, way more than 50% of your nights and weekends, for almost a year, being taken up. After you’ve been to work, had a long hard day, okay, you come home, eat dinner, bore your family for a while, then sit there for six hours writing till the wee hours of the morning…that’s the hardest part.

Q: What are your expectations for the book?
A: I personally always looked at a book as being successful if it goes into a second edition. It must have been good enough to keep selling beyond that first threshold. I think they’re printing between 8,000 to 12,000 copies of my book. I would be really happy if it goes into a second edition.

Q: As a result of writing “Dojo - The Definitive Guide” you’ve had a few new opportunities to share your expertise on the subject. Tell us about being invited to speak at OSCON, The Open Source Conference, and the June article in Linux Journal.
A: I was encouraged by my O’Reilly editor to submit a proposal for a talk, and I would imagine that having O’Reilly care enough to publish a book on the topic in the first place, probably helped some. Getting in to do the talk wasn’t a given, but having the book probably helped. My OSCON talk is on a component of Dojo called GFX. It’s a sub-project of Dojo that allows a developer to do drawing and animation on the screen using one of many backends…SVG, Microsoft Silverlight, VML and in theory you could plug in any kind of drawing backend into it, you write the code according to this GFX API, pick the backend you want to render it with and it just happens. You write the code once and point it anywhere.

I submitted a proposal for Linux Journal last summer. I was just perusing their site and noticed they had an issue coming out about web technology. I thought it might be a good way to get Dojo out there into the mainstream even further than the book.

Q: What have you learned going through this book-writing process?
A: I’ve really come to appreciate just how much work it is. The next time I see a typo in a book I’m going to give the author a lot more slack than I used to.

Knowing technical content is one thing. Being able to communicate is another thing. Being able to communicate technical content is a third thing. Then writing a book about it is entirely different.

Digital Reasoning is fortunate to employ some of the best and brightest minds in their fields and Matthew Russell is no exception.  You can find his book - “Dojo: The Definitive Guide” on bookshelves now. Subscribers to Linux Journal can click here to read Matthew’s article “Dojo: the JavaScript Toolkit with Industrial-Strength Mojo“.

“I personally think we developed language because of our deep inner need to complain.” – Jane Wagner

When it comes to most text analytical tools the only measures given are recall and precision. Some may tie them up nicely and appropriately in the f-measure which is simply the harmonic mean of those two numbers. Usually the discussion of quality ends there and you are quickly whisked off into discussions of functionality, user interface and processing speed.

As I wrote before there are many issues with measuring NLP tools. One of those issues is a lack of accredited measures to apply to them. At DRS we have at the root of our analysis something called the associative network. You can read about how these work in theory and examine a few examples of them. Generally there are a lot of problems with them revolving around their explosive need for memory and the time it takes to process them. At DRS we’ve solved a lot of those problems and find that medium sized corpora work just fine on your standard 2GB laptop. Let me briefly explain what an associative network is, as we’ve defined it.

An Associative Network is a set of related elements from a distribution of elements based on shared features to one or more elements selected from that distribution. Essentially, it is supposed to give you ranked elements that are semantically “closer” to the element(s) provided for comparison. The effectiveness of Associative Networks generally turns on (a) the selection of features of the elements in the distribution to compare and (b) the features of the element provided for comparison that are relevant in ranking. For instance, if I were to provide “fly” as a linguistic element to a set of linguistic elements in a data set, I might want “flying”, “traveling”, and “moving” as my expected association. This, of course, assumes the “sense” of “fly” is as a predicate and not as an entity (such as an insect). If the latter were the case, I might expect the associations to be “insect,” “bug,” and “fruit fly.”

The key above is to recognize features about the elements as used in the data (”fly” as predicate and “fly” as entity would have very different features if properly measured) and which features are apt for comparison (the string “fly” may be insufficient to specify the appropriate set of features to prioritize because its sense may be ambiguous without the user selecting “entity” or “predicate” as a qualifier on “fly”). The ideal Associative Network solves the traditional Natural Language Processing problems of automatic thesauri creation, clustering of semantic nearest neighbors, and brings us very close to effective, unsupervised sense disambiguation technology. Those are some ways that Digital Reasoning applies its Associative Network technology.

This technology is exciting and is very new in commercial grade applications. It is important to understand the strength and weaknesses of this tool. If you were evaluating an analytical tool it would be important for you to evaluate the accuracy of such a system and its utility. I was asked months ago to come up with a measure for Associative Networks. Since I am lazy I went and looked high and low for someone else’s measure first! Sadly there wasn’t anything out there. So I started to analyze what was coming out of our tool. Every attempt failed to produce something I would want to show because the scientific side of me rejected the processes I was developing. The problem was subjectivity. Your hard sciences like physics have unambiguous predictions from theory. The Strange Quark charge is always going to be 87 MeV by the Standard Model and as predictions go this one has always been measured this way in experimentation. When we get to softer sciences things start to get a little more ambiguous and subjective. As I stated in a prior post you have to reject subjectivity as much as possible.

So there I was staring at input terms for the associative net and the resulting list of associated terms given as output. What, therefore, defines good associations for “tree” or “Teddy Roosevelt” or “quark”? When we look at the various ways in which the associative network can be tweeked (there are many variables that control the process) and the fact that different corpora will produce different associations I began to think there was no way to measure this. At least a non-subjective way. Subjectively I can look at a list and using my own knowledge say whether the list “looked right” or not. That is hardly a measure. It certainly isn’t scientific.

Throwing all subjectivity out the window I needed to find a scientific method… I had to make predictions and prove them out through experimentation. Then it hit me. It’s not just the associations. It’s the network. There should be a way of looking at two terms and predicting a third in relationship to both. So, taking an analysts approach I looked at a document from one of my corpora and found that the USS Nimitz has 5,900 sailors and the reactor has a peak-output of 190 MW. Ok, now we are talking. The intersection of associations between USS Nimitz and Sailors should contain 5,900 and the intersection of USS Nimitz and peak-output should contain 190 MW. It seems so simple and yet it eluded me for 2 months trying to solve this problem. I am currently working on a test of this concept and the write-up of the theory. I am sure I will come across some interesting issues and along the way discover more ways of testing associative nets and other semantically related data organization tools. By making these methods open it allows them to be used widely. By making them general in use (this method could be used on a wide variety of systems, including humans) they will have much more Universal applicability. I’ll use this place as my initial forum to announce the results of the experiment and methods one can use to replicate the experiment.

Congrats to our friend and colleague Matthew Russell on the publication of his new O’Reilly book, “Dojo: The Definitive Guide“.

Watch our blog for more information from Matthew on his new book.