Author: gregjordan (Page 1 of 2)

GraphStory helps AgSmarts enable the next agricultural revolution: The Agri Internet of Things

Over the last 200 years, the world has witnessed the agriculture industry make tremendous strides in producing more crops with less labor and overall better agriculture practices. The amount of labor needed to grow 100 bushels of corn has gone from 80 hours in 1850 to 2 hours today. In the same period, the yield for one acre has gone from 40 bushels to over 160 in good years.

Even with these advances in agriculture, farmers must continue to innovate to be successful. Farmers manage their fields while needing to understand dozens of variables, such as changes in weather patterns to the impact of seeding patterns. In addition, farmers in California – as well as other areas of the US – have also met with the harsh realities of drastic drought conditions. To that point, water scarcity is only one of the effects of a rapidly changing climate that is affecting farming.

High Tech in Agricultural Progress

AgSmarts, a two-year old startup based in Memphis, TN, is focused on helping farmers by providing unprecedented and specific crop information that, in turn, lowers their operational expenses and optimizes crop yields. AgSmarts’ Precision Ag includes moisture-sensing technology, predictive analytics, and farm equipment automation that represent an innovative revolution in data-driven agriculture.

Farmers are furnished with smart sensors that connect wirelessly to the cloud and collect & analyze data – from real-time weather data, crop and other variables in their fields. This data helps the crops tell the story about the impact of farm management descisions and changing conditions and, ultimately, can provide direction in order to produce greater yields.

agsmarts-img1

Big Data helps Agronomics

Wireless technologies and the internet of things have made it possible to create affordable field sensors that constantly measure critical data like temperature and soil moisture. Data collected by the sensors is uploaded to the cloud. AgSmart combines the sensor data with weather data, crop growth and many other data points and stores them in a graph database. Graph databases are optimized to quickly create insights from large datasets – turning data into actionable insights.

“Graph Story has been instrumental in getting our cloud application up and running in such a short time. Their hosted graph database gives us all the advantages of a scalable, lightning fast, modern big data solution without any of the hassles of maintaining it and worrying about scalability” says Clayton Plymill – CTO and co-founder of AgSmarts.

In partnering with AgSmarts, Graph Story has helped with the design and implementation of its groundbreaking application as well as hosting and support. Our service allows AgSmarts to focus on their customers and their core business as well as provides an affordable, scalable platform to help further their goals in providing amazing technology in agriculture.

To get started with the Graph Story platform, sign up for our free trial or contact us at contact us with any questions!

_______________________

About AgSmarts: Based in Memphis, TN, AgSmarts is a Precision Ag technology company that offers remote wireless sensing, predictive irrigation and crop management analytics, and equipment automation that collectively represent a revolution in data-driven agriculture. AgSmarts’ platform combines hardware and software solutions into a versatile, powerful and cost effective suite of tools that producers, researchers and agronomic consultants can use today in the struggle to conserve natural resources, control operational costs and maximize crop yields. For more information about AgSmarts, please visit www.agsmarts.com

 

Comparing Graph and Relational Databases

When comparing graph databases to relational databases, one thing that should be clear up front is data affiliation does not have to be exclusive. That is, graph databases – and other NoSQL options – will likely not replace relational databases on the whole. There are well-defined use cases that involve relational databases for the foreseeable future.

However, there are limitations – particularly the time as well as the risk involved to make additions to or updates to a relational database – that have opened up room to use alternatives or, at least, consider complimentary data storage solutions.

In fact, there are a number of use cases where relational databases are often poor fit for the goals of certain data, such as social relationships at-scale or intelligent recommendation engines. Overall, the limitation of how a relationship is defined within a relational database is a main reason to consider a switch to a graph database.

Also, industries of all types are seeing exponential data growth and the type of data that is growing fastest is unstructured. It doesn’t fit well in columns & rows – AKA the relational database schema.

Using a schema-less database model found in graph databases is a huge benefit for applications and the developers that maintain them. If the application can drive the data model, it fits better the with development cycle and can reduce risk when making model changes later.

The relationships in a property graph, like nodes, can have their own properties, such as a weighted score. With that capability, it would be relatively trivial to add this new property on a relationship. It’s especisally useful when the relationahip was not defined when the application was initially created.

For applications that use a relational database, this would be done by creating a join table – as it is known as a in the RDBMS world. This new table joins together two other tables and allows for properties to be stored about that relationship. While is a common practice, it adds a significant layer of complexity and maintenance that does not exist within the graph database world.

Yet another reason you might consider moving to graph database is to remove the work-arounds that must be used to make an application fit within a relational database.   As discussed the previous example, a join table is created in order have metadata that provides properties about relationships between two tables.

Often, new relationships will need to be created, which requires yet another join table. Even if it has the same properties as the other join table, it must be separate in order to ensure the integrity of the relationships.

In the case of graph databases, typed relationships can exist between more than just two types of nodes, For example, a relationship called “LIKES”, e.g.(person)-[:LIKES]->(book), can also be applied to other node types, e.g. (person)-[:LIKES]->(movie).   In fact, the relationship type could be applied between any of the applicable nodes in the graph.

Another reason to consider graph databases over relational database is what can be referred to as “join hell”. While creating the join can be relatively trial, those types of joins provide the least expressive data. Again, applications very often require joins over several tables. It is in this case that the big expense of joins begin to show – in both the development time and the application performance. In addition, if you wanted to modify the join query, it might also require adding more join tables – adding even more complexity to development and worse application performance.

Adding new relationships and the queries that represent them occur at application level. This removes a level development complexity and time and offer better application performance.

While the differences between graph and relational databases, there are a few similarities. A significant similarity is that both can achieve what is known as ACID compliance. It is these set of principles that guarantee that transactions completed by the database are processed reliably, which keeps data safe and consistent.

Why use a Graph Database?

What do LinkedIn, Walmart and eBay as well as many academic and research projects have in common? They all depend upon graph databases as a core part of their technology stack.

Why have such a wide range of industries and fields found a common relationship through graph databases?

The short answer: graph databases offer superior and consistent speed when analyzing relationships in large datasets and offer a tremendously flexible data structure.

As many developers can attest, one of the most tedious pieces of applications dependent on relational databases is managing and maintaining the database schema.
While relational databases are often the right tool for the job, there are some limitations – particularly the time as well as the risk involved to make additions to or update the model – that have opened up
room to use alternatives or, at least, consider complimentary data storage solutions. Enter NoSQL!

When NoSQL databases, such as MongoDB and Cassandra, came along they brought with them a simpler way to model data as well as a high degree of flexibility – or even a schema-less approach – for the model.
While document and key-value databases remove many of the time and effort hurdles, they were mainly designed to handle simple data structures.

However, the most useful, interesting and insightful applications require complex data as well as allow for a deeper understanding of the connections and relationships between different data sets.

For example, Twitter’s graph database – FlockDB – more elegantly solves the complex problem of storing and querying billions of connections than their prior relational database solution. In addition to simplifying the structure of the connections, FlockDB also ensures extremely fast access to this complex data. Twitter is just one use case of many that demonstrate why graph databases have become a draw for many organizations that need to solve scaling issues for their data relationships.

Graph databases offer the blend of simplicity and speed all while permitting data relationships to maintain a first-class status.

While offering fast access to complex data at scale is a primary driver for graph database adoption, another reason they offer the same tremendous flexibility that is found in so many other NoSQL options. The schema-free nature of a graph database permits the data model to evolve without sacrificing any the speed of access or adding significant and costly overhead to development cycles.

With the intersection of graph database capabilities, the growth of graph database interest and the trend towards more connected, big data, graph databases increase the speed of applications as well as the overall development cycle – specifically how graph databases will grow as a leading alternative to relational databases.

PHP[TEK] 2015

We’re super excited to join the PHP community at php[tek] as a sponsor for this year’s conference. Please make sure to stop by our booth in the registration area, grab a game card and t-shirt!

Just in time for tek, we’ve put together a few example apps that help demonstrate the power of graph databases used in conjunction with PHP.

Composer Graph

Ed Finkler, our Lead Dev & Head of Developer Culture, put together an application that covers the Composer ecosystem, specificially the relationships between maintainers and package dependencies. “I’m seriously blown away at how graph databases can easily and performant-ly solve problems that require huge, complex SQL in an RDBMS,” said Ed. “Trying to create these types of dense and complex relationships in a RDBMS is just much, much harder to do. The structure of the data is a perfect fit for a graph.” Check it out at http://packagist.graphstory.com/

php[tek] Twitter Graph

Jeremy Kendall, our CTO, has put together a simple but elegant app that analyzes the Twitter activity around the #phptek hashtag. “Tek is all about the community. With this application, we can analyze and visualize the community conversations taking place on Twitter,” said Jeremy. You can check out Jeremy’s app at http://twitter.graphstory.com/

GraphConnect Europe 2015: (graphs)–[:ARE]->(everywhere)

We couldn’t agree more with the theme for this year’s GraphConnect in London: Graphs are everywhere!

Graph databases are the future, and Neo4J is clearly leading the pack. As a Neo4j partner, we are excited about the road ahead and building out a amazing Graph Database as a service platform.

Neo4j database ranking chart

So to everyone at Graph Connect this week: We hope you’ll enjoy all the sessions and make sure not to miss the closing keynote “Impossible is Nothing with Graphs” by Dr. Jim Webber. You will be sure to enjoy his insights!

Graph Story Presenting at eMerge America conference 2015

Emerge Americas

*** Update: We’ve been selected as one of the 10 Early Stage Finalists! – Presenting Tuesday May 5th – 9.40am!

This week we’ll be presenting in Florida at the eMerge Americas conference. eMerge Americas is a global idea exchange focusing on how technology and innovation are disrupting industries. The conference serves as a platform connecting revolutionary startups, cutting-edge ideas, and global industry leaders & investors across North America, Europe, and Latin America.

Graph Story eMerge AmericasGraph Story has been selected for the Startup Showcase competition. This pitching competition, sponsored by Nasdaq and Google’s Next Wave, has a grand prize of $175,000 in cash investment. After presenting at the SXSW Accelerator competition in January we’re excited to present again and make the case for our Graph Database as a Service business. We’ve made tremendous progress in the last few months and have enjoyed working with several enterprise clients, helping them get their Enterprise Neo4j environment up and running in no-time, all while being fully secure and scalable. So we’re excited to present in front of a jury that includes judges from Palladium, Goldman Sachs and Crunch Ventures – to name a few!

If we don’t see you in Miami, maybe we can meet in Chicago in 3 weeks: Graph Story is sponsoring php[tek] (May 18th-22)!

We have a lot of exciting news in the pipeline, including several new hires – so stay tuned for more!

 

Graph Story selected for SXSW Accelerator Competition

We’re excited to share that Graph Story was selected to participate in the Enterprise and Smart Data Technologies category for the 7th annual SXSW Accelerator competition presented by Oracle.

The SXSW Accelerator competition is the marquee event of SXSW Interactive Festival’s Startup Village, where leading startups from around the world showcase some of the most impressive new technology innovations to a panel of hand-picked judges and a live audience. Hundreds of companies submitted to present at SXSW Accelerator, where Graph Story was selected out of 48 finalists in six different categories.

The two-day event will be held the first weekend of SXSW Interactive, Saturday, March 14 through Sunday, March 15, on the sixth floor of the Downtown Austin Hilton. The pitch competition will then culminate with the SXSW Accelerator Awards Ceremony on Sunday evening, March 15, where winning startups from each category will be announced and honored.

“Over the past six years of companies competing in SXSW Accelerator, more than 50 percent have gone on to receive funding in excess of $1.7 billion and 12 percent of the companies have been acquired,” said SXSW Accelerator Event Producer, Chris Valentine. “This year’s finalists have all demonstrated the capability to change our perception of technology and we expect to see them achieve similar, if not greater, success than our past finalists.”

The Accelerator competition will feature finalists across categories including Enterprise and Smart Data Technologies, Entertainment and Content Technologies, Digital Health and Life Sciences Technologies, Innovative World Technologies, Social Technologies, and Wearable Technologies. SXSW Category Sponsors include Rackspace, Enterprise and Smart Data Technologies Category Sponsor; and IBM, Innovative World Technologies Category Sponsor and Dyn, Social Technologies Category Sponsor.

Moving Beyond the Social Graph: Neo4j Graph Database examples for ad-hoc analysis

If you’ve done any reading on graph databases, then you have likely come across the example of how graph databases can illustrate connections in a social media context. While graphs are excellent at handling social connections, this model just scratches the surface on the variety of use cases that can be addressed within a graph database.

Built-In Ad Hoc Analysis

Graph databases are an excellent match for experimental – or ad hoc – analysis. While there are many features in Neo4j that enable this type of predictive analysis, we will show two built-in features in the Neo4j browser to perform an investigative analysis, including data visualizations and implementing and removing relationships in a graph schema.

Seeing is Believing

When it comes to understanding data, few tools offer the advantages of data visualizations like Neo4j. While the Neo4j browser does offer a tabular view of the data, the visualization features create a more intuitive data representation for data analysts and will reduce the critical time-to-insight factor of data analysis. Consider the following examples of how data can be illustrated with the out-of-the-box utilities in Neo4j.

During a recent graph implementation to help law enforcement analysts, we can see how theft incidents were related by location and time. In a few simple steps, we can illustrate these incidents with visualizations that enabled analysts to summarize the events and make changes to patrol configurations. Here are a few Neo4j graph database examples

usecase1

From the illustration above, we can see how analysts were able to quickly see the difference in theft activity between days (2 on 4/20 & 6 on 4/7 ). Using the same visualization, we changed the presented label of the data to expose the value of the theft activity.

usecase2

From this illustration analysts were quickly able to see that $290 worth of bikes were stolen on 4/20/2014. When we added a relationship between events to not only look at thefts on the same day, but also the same hour, the following visualization showed analysts more detailed information about when these events occurred.

usecase3

From this illustration analysts were quickly able to see that on 11/1/2010 at 2:00 pm, 3 theft events occurred. Finally, consider the following example where a relationship was added to the data to match events which occurred at the same latitude and longitude.

usecase4

The illustration here shows theft events totaling $320 at LNG: -75.059492750000004 and LAT: 40.047509900000001. This data eventually correlated with a geo location database and an exact location was determined to be of high risk and a candidate for increased patrols.

Experimental Connectedness

As demonstrated above, adding relationships like time and space to data is a powerful tool to experimental analysis for law enforcement. Being able to correlate when and where events take place allows these agencies to be proactive rather than reactive to theft events. So let’s take a look at how Neo4j enables this on-the-fly analysis.

With just 3 lines of code, we can add the relationship between theft events based on the theft date property of the theft node.

Create Theft Relationship Based on Date

MATCH (t:Theft), (u:Theft)
where t.THEFT_DATE = u.THEFT_DATE
create UNIQUE (t)-[:STOLEN_ON_SAME_DAY]->(u)

In this example, we first query the graph to find all theft events have the same date and then create a relationship labeled STOLEN_ON_SAME_DAY to the data. In seconds this operation made it possible to correlate these events and produce the visualization provided above.

Create Theft Relationship Based on Date and Hour

MATCH (t:Theft), (u:Theft)
where t.THEFT_DATE = u.THEFT_DATE AND t.THEFT_HOUR = u.THEFT_HOUR
create UNIQUE (t)-[:STOLEN_ON_SAME_DAY_AND_HOUR]->(u)

Just by adding an additional AND condition to the statement for stolen on the same day, we were able to drill deeper into the relationship between theft events and uncover events which occurred during the same hour (represented by the :STOLEN_ON_SAME_DAY_AND_HOUR relationship).

Finally let’s look at how we setup the relationship which correlated theft events which took place at the same latitude and longitude.

Create Theft Relationship Based on Geo-Location

MATCH (t:Theft), (u:Theft)
where t.LAT = u.LAT AND t.LNG = u.LNG
create UNIQUE (t)-[:STOLEN_AT_SAME_LOCATION]->(u)

Again with a simple where statement, we were able to link events by geo-location and represent the relationship STOLEN_AT_SAME_LOCATION.

Looking Ahead

From these cases, it should be clear that graphs can offer two very easy ways to analyze more than just social media data. After all, being able to relate data points by spatial and temporal elements is applicable to more areas than just law enforcement. From a retail perspective, this can lead to illustrations of product demand by store location and time of year. Apply the same relationships to customer purchasing, and you can gain insight into when and where customers are buying what products.

When you factor in the relative ease and speed at which this type of analysis can be performed, using graphs to perform analysis is a low risk/high reward exercise.
While social media projects get much of the limelight in graphs, the real power of graphs is being able to move beyond relational database constraints and putting the analytical strength of Neo4j to use.

with use case work and code provided by William Sharp

Graph Connect 2014

For the uninitiated, Graph Connect is a one-day event hosted by Neo4j (along with some awesome sponsors) and includes tracks on graph use cases as well as training for graph enthusiasts.  It started in 2012 ( and not 2011 as some might believe ) and attendance has doubled each year. This year over 700 graphistas attended the conference at San Francisco’s Jazz Center, and, for those that could not attend, we certainly missed you and hope to see you next year. The 2014 conference included great presentations that provided an in-depth view into interesting use cases, a chance to view the current graph landscape and see where the community will go next.

graphconnect-5443

2014 Keynote

Emil Eifrem’s keynote focused on the fact that “graphs are eating the world”, which is a take on Marc Andressen’s statement that “software is eating the world”.   For example, before (and during) the rise of Paypal, a number of financial orgs to tried to create their own large-scale payment platforms. One stumbling block – a big reason that many of these companies didn’t succeed to Paypal’s degree – was their inability to effectively combat fraud.  Paypal developed analytical tools and the capability to more closely monitor their payment network.  As Emil noted, Paypal was able to effectively leverage the graph-shaped data to ensure payments were handled with fraud management as a front-line tactic.  Network analysis, which in the case of fraud analysis might start with person-[:COMMITS]->transaction, is a sweet spot for Graph databases.

graphconnect-5486

In addition to specific use cases, Emil discussed the five types of Graphs – Social, Intent, Consumption, Interest, & Mobile – as defined in a recent report by Gartner and demonstrated that “graph analysis is possibly the single most effective competitive differentiator for organizations pursuing data-driven operations and decisions after the design of data capture.”

The specific mention of “decisions after the design of data capture” is one reason why graph modeling and analysis is so effective in delivering better data insight.  Graphs can allow organizations to (1) ask the right questions from the data and (2) those questions don’t have to be precisely identified prior to data capture.  Finally, Emil noted that while 2.5% of enterprises reported using the power of graph databases in 2014, Forrester analysts estimate that over 25% will be using them by 2017.

Kurt Freytag - Head of Product @ CrunchBase

Use Cases

Kurt Freytag of Crunchbase covered “The Business Graph” and discussed some of the reasons why they moved from MySQL to Neo4j. Two of the big reasons were that graphs are a natural way to model data and graphs adapt easily to changing requirements. As Kurt mentioned, with a Relational database “you’re flattening the real world into a model that fits narrow limits and isn’t completely integrated across the developer thought-process”, but with a graph “there’s no need to shift contexts in how you think about data”.

Similar to the advantages of a graph referenced in the Gartner report, Kurt talked about the built-in business intelligence that comes with a graph database, which allows for both very specific questions or very general questions “when you don’t know the questions in advance”. In addition, Kurt discussed the speed of development that’s possible when using a graph database, specifically noting that Crunchbase Events @ #TCDisrupt London was built in just 2 weeks. Developing the same application using an RDBMS “would have taken two months in the old version of Crunchbase”.

Aaron Wallace, global product manger at Pitney Bowes, discussed their program aimed at master data management, which brings together a multi-dimensional view of the customer. Pitney Bowes started with graph database in 2010 and cited multiple platform and language support, high performance, and acidity as key to selecting Neo4j.

graphconnect-5548

Volker Pacher, Senior Developer at eBay Local (formerly Shutl), started off his talk by placing an order on eBay Local to demonstrate the increased speed of their application backed by Neo4j. The delivery showed up just before he completed his talk – less than 30 minutes later. In the interim, Volker reviewed Shutl’s switch from MySQL to Neo4j, discussed the ease of learning and using Cypher (Neo4j’s declarative query language), and how modeling domains turned from being a chore into a fun exercise.

graphconnect-5573

Stephen Henderson and Shashank Tiwari of Elementum shared their experience managing and configuring Neo4j and pointed out that tuning doesn’t go away: “If you’re building a system and you ignore tuning, I have to question whether your building a serious system”. Some other lessons learned from their project, specifically;

  • modeling took many iterations (60% to 70% of time should be spent on this)
  • business rule validations can be tricky
  • leverage the traversal api early

graphconnect-5554

Wrapping up

There were hundreds of great insights at Graph Connect, but Jim Webber’s succinct reminder of how we all got here is one that stood out: “complex data never fit into the RDBMS model. RDBMS were designed for simple, predictable data.”

graphconnect-5618

That message captures why new developers will continue to turn towards graph databases to solve complex data analysis. It underscores the tremendous divide we’ve crossed between the way we used to build apps with our data and the new, more efficient and faster possibilities with a graph.

GaaS – Why Graph Database Hosting Makes Sense

In the past decade, organizations have made two significant shifts in data management practices, specifically where and how data is stored.

In terms of where data is stored, cloud-based services have revolutionized the way organizations manage software and hardware infrastructure – bringing with it cost savings, convenience and reliability.

The NoSQL (Not Only SQL) movement has similarly enhanced how organizations manage data. NoSQL has allowed organizations to blend traditional data structures with newer, more specialized and often faster types of data storage solutions.

Both cloud-based services and NoSQL offer significant advantages and benefits to organizations. However, the benefits increase exponentially when combined as a single strategy by improving scalability and responsiveness, increasing data insight and lowering costs.

Service-based Infrastructure

cloudpicWhile service-based solutions have enabled organizations to change where data is stored, it is more than just a change in location. One of the most compelling benefits to this model is its affordability. In the service-based model, you only purchase what you need, specifically storage size and duration of service.

Service-based models can also significantly decrease the time to market. Do you need to build out a special project that analyzes a marketing campaign’s effectiveness? In a traditional on-site solution, you will likely be ready to launch the next marketing campaign by the time new hardware has been ordered. However, a service-based solution allows necessary resources to be set up within a few minutes and have applications deployed in the same day.

This elasticity allows organizations to decrease procurement risks due to lower up-front costs and increase flexibility by taking advantage of the pay-as-you-go cycle of service-based contracts.

Service-based solutions also offer a benefit when budgeting for new projects and addressing changes in organizational needs. When a business utilizes a service-based model, technology costs can be shifted from a Capital Expense (CapEx) to an Operating Expense (OpEx).

While service-based offerings are a great way to add scale and decrease the hardware footprint, the financial benefits can deliver an immediate competitive advantage to organizations of all sizes.

Another way to add a competitive advantage is to quickly and easily implement new technologies that deliver insight to business users.

Connected Data – Graphic Insight

The focus of using data should always be to learn something. The more logical this learning process is, the faster the learning will occur. These decreased learning curves can lead to a competitive advantage.

The underlying structure of a graph database is a very natural, relationship-first structure, which enables developers to deliver solutions that are better aligned with the business more quickly than with traditional database solutions.

Take for example the graph database model below, which translates business logic into the base structure of a graph solution.

graph1

Here is a list of statements gathered from a business user which were used to define this model:

  1. A customer purchases products
  2. A customer lives at an address
  3. A customer can be reached at various contacts
  4. A customer or prospect can live with another customer or prospect
  5. A customer or prospect can be related to another customer or prospect
  6. A product is included on an order

Now let’s look at the visualization of this model once data is extracted from a CRM application and stored in a graph database.

customer_placed_order_contains_product

In this model, customer records are represented in orange, order records are represented in yellow and product records are represented in purple.

From this we can determine that customer #2 and customer #8 placed several orders containing the same product. We can also determine that customer #2 and customer #8 are related.

customer_placed_order_contains_product_closer
In minutes a business analyst can review this graph visualization and identify an opportunity to better serve a household of customers by offering bulk purchasing deals.

In short, this organization can learn and react more quickly because graph databases make it easier to create and visualize relationships.

A Graph Database Hosting – An Affordable Way to Quickly Gain Insight

When you combine the affordability and budget-friendliness of service-based solutions with the relationship-first design and speed-to-insight of graph database solutions, you have a low-risk, high-reward engine which can deliver competitive advantages in a short timeframe.

By leveraging the flexibility, affordability and speed of Graphs as a Service, businesses achieve a better understanding of their data and can deliver greater value to their customers. Try our Neo4J based hosting today!

with William Sharp & Jeremy Kendall

Page 1 of 2

Powered by WordPress & Theme by Anders Norén