One of my favorite projects recently has been the ONEsite Analytics & Activity Dashboard which has added many new features and is under active development. Because we come from a web hosting background we have always been very interested in capturing as much information as possible about both the traffic on our sites and the actual health of our servers and their public and private services. As a very client-focused company we have strived to provide as much information as possible to our clients through our Control Panel.
Statistics have been an important part of the ONEsite platform from the very beginning. We offered robust statistics on blog traffic when we first launched the ONEsite blogging capabilities many years ago. As our traffic and number of incredible customers has grown over the years we have continued to gather large amounts of data about the sites running on the ONEsite and present this information to our customers in the Control Panel for their network.
We began a project in Q4 last year headed by Evan Stenmark to modernize the way we collect and log data as well as collate, analyze and package as much as possible in the Control Panel for our customers to use. We see this as a key initiative because the platform generates much data which is key to understanding the activity of the community and its members and how it grows both organically and virally. Many of our customers use 3rd party Analytics such as Omniture or Google Analytics to track pageviews or referrals; however, the ONEsite servers actually collect much more information than is available by these 3rd party companies. In order to describe how I need to describe a bit about our backend architecture.
At ONEsite we use PHP extensively and developed a PHP Event System as part of our core architecture several years ago. This event system is a key element in everything from our user content generation flows to our moderation capabilities and also allows for a lot of our unparalleled customization and integration capabilities. Our event system was inspired by many of the excellent capabilities within ActionScript 3.0.
Essentially, all user interactions on the platform trigger events in our backend code which may in turn trigger other events. Thus, if a user writes a blog post in our system it may trigger an event such as POST-PUBLISHED. The platform has the capability of attaching multiple event listeners to an event which may in turn run core features within the platform or custom features enabled for single networks. Thus, everything from blog post premoderation to blog post pinging to custom data warehousing about blogs may be achieved by attaching event listeners to the POST-PUBLISHED event.
The ONEsite platform utilizes hundreds of distinct events to power all of our bountiful features. Networks on the ONEsite platform generate hundreds of millions events on any given day. Knowledge about these events is very useful for analytics purposes and thus we decided to tightly integrate our events system with our analytics capabilities. In order to do so we began to log events which occur on the platform and all relevant event metadata about that specific event or metadata shared by all events.
Because this generated a significant amount of information we decided to heavily invest in a new backend architecture to store all of this data. We chose to use an open source technology, Scribe, to log all of the events on the platform. Scribe was originally developed by Facebook and has been key to their scaling to hundreds of millions of users.
Scribe allows us to store and organize massive amounts of information collected by the servers in our datacenters and to access that information historically. However, in order to actually generate value from that information store we needed to use other tools. Principally among these tools is Hive which as their wiki says: "a data warehouse infrastructure built on top of Hadoop." Hive allows us to run queries ranging from "How many forum posts were made this week compared to the same week last year?" to "Do users who choose Blue as their favorite color upload more photos than other users?"
Hive, along with other Hadoop tools and custom code, generates the data which is presented to networks in our Dashboard. The ONEsite Dashboard allows network operators to quickly see the overall health of their network as well as run reports for marketing campaigns or other initiatives. We are continually working on more advanced capabilities in the Dashbaord to give network operators access to more network vital information, more demographic analysis of its users, and more trends occuring within the community.
Understanding both who the end users of the ONEsite platform are and how they use the tools we provide is key to the success of our company and the sites we run for our clients. We believe our Event system and our Analytics Dashboard provide ONEsite customers with valuable and necessary information to achieve their community goals and as such ongoing development into Analytics is a key item on our roadmap.
