How to Analyze Data with Emcien

Analysis with Emcien

If you're familiar with Emcien's software and would like a more detailed tutorial of how we work, click the links below for a walkthrough.

Data Prep

While Emcien offers multiple file formats, the one named “wide format” is the more universal and most commonly used Emcien format. You can use the wide format for multi-dimensional data, such as demographics or configurable products. In the wide format, each transaction is identified by a single row of data.

An example of the wide format is displayed below.

This example contains four transactions, each represented by a single row of data. In this data, each transaction represents a client and their associated demographics data.

Unlike the receipt format, the wide format has no required columns and supports user-defined columns.

Wide Format Details

Headers are required for each column in the wide format. Your file must contain at least two user-defined columns with a maximum of 1,000 total columns.

While the data is allowed to contain any UTF-8 characters the header must be in lower ASCII.

For all the details on the “Wide” and other data formats visit article for 

Depending on your data, Data Prep can include the following steps:

Automated Analysis

To begin, let’s go to our home page and click on ‘Analyze Data’

Next, select the file we want to analyze and scroll down

Now, if you are running a prediction analysis, enter the category for which you want to extract predictive rules for (ex. ‘Sale’ for the sample data file for CRM Sales data). If you are just trying to find relationships within the data without making predictions, leave this field blank and hit ‘Analyze’.

The engine will have animated steps displaying what it’s doing as it tokenizes, connects, and ranks the data. The picture below describes what is happening in each step.

Now that the data has been analyzed, it's time to see the analyzed product. Scroll to the top of the page and click on 'Review Results'.

Review Results

Because Emcien analyzes all of the data, the results reflect a holistic view of your data and the patterns within it.   Here is a quick breakdown of the some of the main results you will be provided at the end of an analysis.

Dashboard

This is the home page of Emcien Patterns. The colorful chart in the middle is a visual representation of how the data is connected, and gives us an idea of what the engine found in the analysis. To explore the connections within the data a little, let’s go ahead and click on the ‘Clusters’ tab at the top of the page to learn more.

Clusters

The Clusters page begins to tell us the story of how our data is connected. The blue boxes at the top represent categories, and are there to show you how items in the selected category are connected with items in other categories. In the picture above, we have selected the category “Age” and we can see how many clusters items in that category make with other categories (a cluster is a connection between 2 or more items). Clicking on a ‘Tell Me’ Button will present each row in a more readable format. Let’s scroll down to find out more about this category.

Further down the page we see the individual connections for items in this category. Each line represents a connection, where the item on the left suggests the presence of the item on the right. So for the first row in our example, when the item “[17.0-32.0] Age” is present in a row, it suggests that the item “single marital status” will be present in that row as well, with a conditional probability of 100%.After reading a few more lines to get an understanding of this category, let’s scroll back to the top and select the ‘Sale’ category to learn more.

Category Details

After clicking the Sale category, we can see the connections that lead to a ‘Yes’ or ‘No’ for the sale. Clicking the arrow inside of a ‘no’’ box will allow us to go to the category detail page to learn more about the different outcomes of ‘Sale’.

The category detail page shows us the different values for ‘Sale’ that were found in the data set. To learn more about the patterns associated with a successful sale, click the ‘yes’ item.

Item Details

The item detail screen shows us the other items that are most connected to the item we’ve chosen. What this means is: For the selected item (in this case, ‘yes’ for sale), which other data values occur in transactions most often with our selected item. By seeing this, we can start to see the relation of our item to other values, and get a sense for what is held in the data. To understand this view a bit more, click on ‘Explore Graph’, as seen in the bottom right of the picture.

Here is a graphical representation of the previous screen, showing our selected item (‘yes’ for sale) and all of the items it is most strongly connected with. by highlighting individual items we see the connection patterns and can begin to understand the data within our data set.

Predictive Rules

Now that we know the patterns and relationships within our sales data, we have finished our first use case. Now, if we'd like to use those connections to find the unique predictors for what causes a successful sale, clikc on 'View Predictors' on the right hand side of your screen.

The Predictions screen shows us the individual combinations of items that, when found in a transaction together, yield a high probability of our predicted item being present in that transaction. Think of these as rules- when the items on the left are found in a row, there is the displayed probability that the item on the right will be present as well. To download a list of these rules, click the ‘Download CSV’ button in the top right.