Tag Archives: BI

Why BI in the Cloud?

The Cloud

The cloud opens up all new options for companies looking to either build out their own solution or to leverage new products that are native to cloud. There are currently three generally accepted cloud delivery models: Software as a Service (SaaS), Platform as a Service (PaaS) and Infrastructure as a Service (IaaS). I consider SaaS and IaaS to be the models that will be the most effective for BI.

One of the concerns that are often raised regarding the cloud is security – this in particular with BI since it involves data and sometimes personal information. The question then is, how will a provider that specializes in managing a data center (in the case of IaaS) or large-scale application and data hosting (in the case of SaaS) any less secure than a company who’s focus is on their own business and not on securing the data center? You will need to verify a few things depending the delivery model that you are using. For SaaS, you will want to ensure that the provider can give you secure login and authentication, the ability to define granular levels of access, SSL, and the option of encryption at rest. For IaaS, the customer is typically responsible for setting up security but you will want to see ISAE 16, ISO 7001 certification, DDOS mitigation as well as options to provision firewalls and load balancers. The Cloud Security Alliance (CSA) is currently working on their Security, Trust and Assurance Registry (STAR) which will make it easier to determine security criteria for a cloud provider but it is currently only in the preview stage.

Deciding which Cloud Model

If you are interested in just providing data and then having software available to analyze, manipulate and create reports, SaaS products may work for you. Example of some SaaS BI companies are BIRST, PivotLink, and GoodData. All of them provide the BI-stack, which is a method for extracting, transforming and loading data, a place to store the data, and a front-end to create ah-hoc reports and dashboards. The advantages of using this style of BI is that all the management of the infrastructure and software is handled by the vendor. There is also minimal up-front costs – the model is to simply pay for what you use.

If you want to build your own BI platform, you can leverage IaaS and open-source software. You will need to find an IaaS vendor that best suites your needs – Amazon Web Services, GoGrid and Rackspace are the leaders in this space. Using IaaS, you will have full control of your infrastructure – you can determine how many servers to spin up, the security you want to use and how you want to store the data. You will also need to build out the software assuming that you have that expertise in-house. Some good open-source options are Talend for data migration and manipulation, Pentaho for data integration and analytics, BIRT for reporting and MySQL or Postgres for the database. This model requires more system administration expertise and developer resources to build out a custom BI solution but this may be worth the investment if you want tighter control of your product or have very specific custom needs. In either case, if you can leverage the features offered in open-source software, your will also minimize up-front costs and will pay for the infrastructure that you use – with the option of spinning up servers to meet demand or remove servers when they are no longer needed. You can start off with small 1GB server and expand to more cores, more RAM and more storage quickly and easily.

In addition, the flexibility of the cloud gives you the option to expand your infrastructure if you need to incorporate a Big Data solution to meet a particular use case. Currently, most of the popular technology is open-source such as Hadoop, MongoDB and Cassandra. I discuss Big Data in more depth in a previous blog post.

Ultimately, you may decide that you need all the features that are offered by a traditional BI vendor or have already made the investment in a particular infrastructure or technology. After all, these companies have been around a long time and there are many talent individuals who are well-versed in these products. However, if you are interested in lowering costs and off-load the infrastructure and software work to another vendor or are new to BI and want to get started with minimal up-front costs, the cloud based BI solutions might be the right option for you. Instead of having to project growth in order to order the hardware up-front, you will have the ability to pay-as-you-go, and add infrastructure and cost only as your growth demands. DASHbay is experienced in both delivery models discussed here and we can provide the right analytics expertise and development experience for your BI needs. Considering the growth in data, the flexibility of the cloud and the much needed analytic features of a BI solution work well together to provide a powerful, low-cost and scalable solution. Make sure that you work with the right vendors and partners to make your project a success!

Pining for Pinterest

What does the Web’s hottest social network hold for data analytics?

Pinterest recently broke 11 million users this year. It was reportedly the fastest social network to reach 10 million unique users since its launch of closed beta in March 2010. However, that is somewhat in dispute, with many claiming that Formspring is still holds that title. Either way, there is no debate that Pinterest is growing at a phenomenal rate.

What is Pinterest anyway?

Since Pinterest headquarters is just down the Peninsula, I caught on to this site relatively early. However, a recent dinner with some out-of-town colleagues reminded me that not everyone obsessively follows startup news like I do.

Pinterest is a social network that links people together by their interests rather than their social circle. So unlike Facebook, which resolves around your friends, on Pinterest people express themselves and find friends through common interests. Another twist is that this is done via a metaphor of a pin board. Users “pin” photos that they find on site, from around the web, or uploaded directly to the site onto user-created “boards”. Users have boards to organize their pins into categories like “Travel” or “Humor”.

Since Pinterest is picture based, visual-type content show far better. Infographics, for example, are far more represented there than articles (some of which have no pictures to even pin). Brands are starting to take notice – Life magazine has a large following, posting mostly archived photos. Retailers are particularly strong here, since most of the items sold there are physical and therefore lends itself well to a pictorial presentation. In fact, Pinterest is driving more referral traffic than Google+, YouTube and Linkedin. Combined.

So who is on Pinterest?

Modea, a digital advertising agency compiled some demographic information on Pinterest users and the results were quite interesting. On average, users spend 15.8 minutes “pinning” while Facebook users only spend 12.1 minutes liking things. Almost a third of Pinterest users have an annual household income of at least $100,000 and almost 70% are women with the majority aged 25-34. So young, upper-middle class and female – sounds like the demographic that advertisers will want to target. Of course, that is just the is the US. The demographics in the UK are decidedly different – 56% male, 29% in the highest income bracket and interested in venture capital (go figure).

However, this doesn’t mean that there aren’t a good mix of interests on Pinterest. The Board of Man which is a board focused on more “manly” interests has 220,000 followers. Even the US Army has gotten into with its own set of boards which is managed by their Chief of Public Affairs.

It’s all about the data

Pinning is fun and all but the real value is the information captured by all those young and wealthy consumers. It’s also capturing a demographic that isn’t as prevalent on most of the social networks (meaning mostly male and on the coasts). Users are specifically stating their interests – in particular things that they covet or plan to purchase. In addition, the content is persistent. Unlike Twitter or Facebook, your pins stay present on the board for you to easily eview again later (and don’t come back from the dead as people discovered during the Facebook switch to Timeline). Retailers are starting to take notice – creating their own pages and engaging with users with interesting content.

Pinterest is still in beta so it has some things that I think it needs to add to become more robust.

It really needs some sort of robust and standards based API (like REST). This will give websites the ability to better integrate with the social network. It also creates the capability for 3rd parties to start extracting some interesting data.
An easier way to find people. It’s all about the interests and not friends but still, it’s impossible to find someone if they aren’t already friends with you on Facebook.
Allow some pins to stand out in some way (either promoted or simply popular)
Work in some additional content other than photos. While photos really enrich the site, having the ability to have music or text add additional context to the photos would be helpful.

In particular, an API in conjunction with other APIs would have the ability for developers to create some interesting apps in combination with Facebook, Twitter or Foursquare. The mash-up of the Facebook social graph with the Pinterest interest graph would be quite interesting. Since the information posted on Pinterest is public by default and generated in real-time, it would be a good indication of the zeitgeist in terms of interests, which would be a welcome addition to the topic trends already occurring on Twitter.

Customer retention metrics

Last night (Tue July 19th), I was fortunate to be able to speak to the SVForum Business Intelligence special interest group (SIG).

After introducing the audience to DASHbay, I took them through an implementation we did using our Quick Analysis practice, which leverages open source software (especially BIRT and postgresql), cloud computing (on AWS), and rapid, iterative development.

The implementation itself was a dashboard, built with BIRT in less than a week and showing metrics for account acquisition and retention. The metrics help any business track not just how well they are acquiring customers, but how well they are keeping them.

Our customer was able to get at the metrics via a URL to a server running in the cloud, set up just for them. It’s a great way to leverage cloud computing: no IT procurement costs or delays, and you only pay for it while you need it.

We talked about DASHbay’s Report Server product, which among other features, allows us to capture any useful piece of the report, and include it in any web page. It also provides permissioning and authentication, taxonomy for organizing reports, and more.

I got an excellent reception from the audience, and was pleased with the reaction and discussions afterwards. Thanks to all who attended!

If you didn’t get a chance to be there, please get in touch so we can talk to you more about our Report Server for BIRT, our Quick Analysis Service, or many custom BI and Data Analytics services. Customer Retention is one very useful application which we can provide, but our tools and techniques are applicable to most common business analysis problems.

Terry