Nati Shalom's Blog

Writing your own real-time search engine and Adwords service

There are various ways to implement a search engine. The typical way uses a crawler technique. The crawler continuously scans the information, collects changes and indexes them. The indexing service is a centerpiece in the architecture and enables quick matching of keywords into search results.

In some cases this technique is not applicable. For example, imagine the case of eCommerce applications. When you submit a bid on an auction, you expect to see it immediately popping up in the search results.This calls for a rather different architecture then the one typically used for Internet applications. The key is how fast we can put new data into the index server. Sounds simple, right? Well, most index servers are highly optimized for fast read, however they tend to be quite heavy on write operations.

This is where it makes sense to put the index server in-memory. Having an in-memory index enables both fast writes and fast searches. There is one small caveat, however – memory is limited in capacity and is not considered reliable, i.e. if the memory fails the data stored in that memory is gone. This is where In-Memory Data Grids come to the rescue. An In-Memory Data Grid (IMDG) addresses the capacity and reliability of memory. Capacity is addressed by breaking the data into multiple partitions; reliability is achieved by having at least one copy of each partition available in another memory instance. This is exactly what Shay Banon did in his Compass project, where goes into additional detail about how this model works.

Quoting Banon from his excellent post: Collocated Indexing and Distributed Search with GigaSpaces

"This type of integration takes collocation of indexing and searching to a new level. Indexing and Search operations are performed in a collocated manner in memory making them extremely fast. Scalability is easily handled by adding more partitions, and high availability is provided by adding backups to each partition"

Me and Shay had various discussions about that model. I personally see great potential in this type of offering. With the move of many of the batch analytics application into real-time-analytics, real-time search will become more common for many applications.

Using the search engine as an advertising channel - Adwords

Once you got your own search running, you’ve created a potential commerce area without even noticing. Different suppliers can compete on specific keywords and on the real-estate (location on the page) they'll get on the search result (first, second, up, down..). Unlike TV commercial that sell ads on specific timeslot, Adwords provides an advertising channel per search click! One common way to take advantage of Adwords is to use Google Adsense which enables you to put Google ads in your site and get rewarded for the clicks that your site is generating. Having said that, the limitation of this model is that it relies on on the fact that you’re using Google as your search engine. If you are looking to use real-time search that is tailored to your site, this is not going to work. So if you’ve already gotten to the conclusion that you need your own custom search engine, you might as well implement your own Adwords to drive more money out of it.

Adwords is a very interesting model – it’s basically a sophisticated biding system. The algorithm involves matching between the supplier’s phrase and the phrase in the search query. The matching first looks at those bidders with sufficient budget on their account, it then looks at the track record of each supplier and rates the supplier based on the number of click-throughs that they got on their ads. For those who were selected, there is another algorithm the determines the order and specific location in which the ads appear on the screen. One of the most trivial one is exact match of keywords, i.e. ads with exact match appear higher up in the list.

The challenge with implementing your own Adwords is that this matching process needs to happen in real time during the search. So how can this be done?

If we consider Shay’s comment above, it should be fairly clear. If we already have the index in-memory, and search can be executed within the index server itself, why can't we use the same idea for matching Adwords?

Well, the idea behind Space Based Architecture really came from these type of bidding scenarios. We realized that executing the matching algorithm collocated with the data is going to speed up the matching time significantly and improve response time. Below, I tried to sketch a simple diagram that illustrates how this model would work.

The indexed data will be stored in one set of partitioned cluster and Adwords data will be stored in another partition.

The search services will be running collocated with the index service as Banon suggested. In similar way, the Adwords matching services will be running collocated with the Adwords data.

A search request gets to the search portal. The search portal executes the search query as well as the Adwords matching in parallel.

It uses a map/reduce execution, meaning it is able to aggregate all matching results from each partition.

To enable this type of operation you can use either the Service Virtualization Framework in case you'd like a strongly typed query service, or the executors framework in case you’re looking for ad-hoc tasks execution.

You can use Futures as the return value in order to simplify the process of executing the search and Adwords matching in parallel. Future will enable you to fork those two tasks and then collect the results using that future handle.

If you would like to process all the results as they they arrive, you can use AsyncResultFilter for that purpose (a typical use case will be to print the result on your page while still waiting for the rest of the result to arrive). For more information and options on using parallel execution to speed up the search and matching process, refer to the executors framework.

A real-life scenario

One of the recent real-life scenarios that I came across recently is Rednano. Rednano is a local portal in Singapore that provides localized search capabilities. Interestingly enough, they built their portal on Spring and Hibernate. They also looked for a way to build their new add-on services such as Adwords in a scalable manner, and GigaSpacs was pretty much the closest solution they could get to meet their needs.

According to Patrick Ng, CTO from Rednano, they where able to integrate GigaSpaces in their existing Spring architecture in matter of two days! Patrick provides an interesting insight on their selection process in an online presentation during a Cloud event in Singapore.

Rednano - The Decision For Selecting Partner - Technology

View more presentations from gigaspaces. (tags: rednano technology)

During the writing of these lines, I came across some interesting news: Rednano wins award for breaking new ground. Quoting from the news item:

Local search and directory engine rednano.sg has won a prestigious international award given to companies with effective online search technologies.

The search engine, a collaboration between Singapore Press Holdings and Schibsted ASA, won the Digital Market Award at the Fastforward'09 business and technology conference in Las Vegas earlier this week.

The annual award is given to companies which constantly 'create new and value-added services for its users' in the fast-moving technology world.

Congratulation to Rednano on their success!

Have we learned the lesson from the recent economic meltdown?

I got an interesting cartoon from a very close friend, which I felt was worth sharing because it very simply explains what's wrong with the current way of thinking in many areas of the industry. Instead of encouraging criticism and accepting it in a constructive way we often fight back, arguing that "you must be stupid" not to see how great we are. When we finally realize we were wrong, we'll blame you for causing us to lose many jobs, and if that's not enough, when we're finally out of business we'll ask you to subsidize us so we can survive all the mistakes we've made! Sounds familiar?

Obama attacks Wall Street's executive really makes me wonder.. are we really going to learn the lessons from this crisis?

Oh in case you wonder this cartoon was drawn on the 2/16/1992  :)

Simple and cost effective Enterprise-SOA with GigaSpaces and Mule

MuleSource is a rather popular open source ESB implementation. What I particularly liked about it when I first saw it is how simple it is to glue different services that are communicating in different protocols in a very simple way even if they don't comply with the same standard API.

Another thing that I like about MuleSource is its light weight and simple architecture. Unlike many other ESB implementations, Mule does not bind itself to the use of web services. In fact you can use Mule for orchestrating the flow between SAP and Siebel in the same way that you can orchestrate two POJO services collocated under the same process. This approach has made the integration with GigaSpaces possible as it has enabled us to introduce many of our innovative ideas and bring in the value of Space Based Architecture into the Enterprise SOA world. It is also a good fit with the type of customers we are engaged with in where latency and performance play an important role. The interesting thing is that thanks to the Mule connector architecture your application code does not need to “know” anything about GigaSpaces. All it "sees" is better performance and reliability. For GigaSpaces users the integration with Mule helps to connect GigaSpaces applications to the external world quite easily and leverage many of the existing sets of connectors available with Mule to plug-in those other protocols without writing additional code. We already have a few customers who chose to use GigaSpaces and Mule together and are quite happy with the results. An example of some of the feedback being received can be seen in the following post GigaSpaces and Mule= Pure Sex.

GigaSpaces and Mule ESB integration overview:

Basically there are a few components to the integration: 

• GigaSpaces connector – The GigaSpaces connector enables GigaSpaces to plug-in as an high performance transport.
• Mule server clustering – This is where GigaSpaces clustering is leveraged to enable in-memory fault tolerant solution for the Mule server itself.

The GigaSpaces Mule Connector:

The diagram below shows how services that are running within Mule can use the GigaSpaces Connector as a transport layer.

As can be seen in this diagram the services are unaware of the GigaSpaces implementation. These details are abstracted by the Mule server. The interaction is done declaratively outside of the services code. When a service receives an event it processes it and the result of that event is sent back to Mule which in return calls the GigaSpaces Connector to store that state. Other service uses the same connector to wait for that state change and once that happen that triggers the appropriate method on that service implementation and so forth.

Mule server clustering

The diagram below illustrates how Mule integrates with GigaSpaces in-memory cluster to ensure high availability and reliability.

 

In general all the Mule state (SEDA Queues in particular) as well as the application state is stored in the GigaSpaces in-memory cluster. That state is replicated synchronously to a backup instance. When the primary node fails the back--up node takes over automatically and continues to process the events from the exact point where they failed. This enables continuous high availability of the application and seamless fail-over without enforcing the user to take explicit action during that failure event and without the need to handle complex re-tries to ensure the reliability and consistency of the application after a failure event.

Announcement:

Given the positive experience both us (GigaSpaces) and the MuleSource team have had, we decided to formalize this integration. . This Wednesday (Feb 11 @ 9 AM PDT/ 12 PM EDT/ 6 PM CET) we are hosting a joint webinar in which we will unveil the details behind this new development. In this webinar Uri Cohen, GigaSpaces Product Manager, and Ken Yagen, Sr. Director of Engineering at MuleSource, will discuss the details behind this integration and use a live demonstration to illustrate how it works. This will also be a good opportunity to ask the experts more questions about this joint offering. To join this webinar make sure that you register prior to the event here

See you at the webinar!

What is the cloud? An end-user view.

There is a continuous debate surrounding THE definition of cloud computing.
Is it just another name for Virtualization? Is it a “re-packaging” of Grid Computing? Or is it just the familiar hosting services we know and love with an additional (and more sophisticated) layer of automation?
To be honest – I think it will be very hard to finalize this debate conclusively by just sticking to high-level terminologies. As technical folks we often tend to dive into technical debates and in the course of doing so, we loose sight of the reasons that brought up this debate in the first place. The discussion about the definition of cloud computing is no exception.

Instead of continuing the endless debate, I thought that taking an end-user view would be much simpler to agree on. Here is a snippet from a response that I gave on one of the recent threads on the cloud mailing list: Is "Internal Cloud" an oxymoron? that tries to address this question:

Based on my experience there are two main driving forces for Cloud-Computing:

1. On demand computing i.e. the ability to get a resource when I need it in matters of minutes.
2. Pay-per use i.e. the ability to pay only for what I use.

The rest is implementation detail. From an end user perspective it doesn't matter that much if I can get the above value from an internal or external cloud.

In my view what we’ll see in the near future is a hybrid-cloud i.e. local-IT + Internet Cloud. A typical scenario would be to run testing on
public cloud and production internally. There are many other scenarios that justify this type of hybrid model.

In the long run, the only real question about internal clouds is whether local-IT would be able to compete with the cost and level of service of the public cloud. In my mind, this is where economies of scale play a critical role (See James Liddle’s recent blog: The Economics of Cloud ). i.e. the bigger the data-center is, the higher the chance that the cost and level of service will be better.
It may very well be that over time, the public clouds will become the only option. Just like electric energy. In fact just like electric energy, there will be those who will “generate” their own power, but will develop it as a hybrid. Sometime they pull resources from the public “utility”, while at other times they use their own “utility”.

This thinking was later enhanced by Miha Ahronovitz in his response on another thread here.

Understanding the cloud-stack by analogy

Another way that I found useful when I need to explain the cloud-stack is by analogy to the existing application stack:

The above diagram is a very simplified view of the world. It explains (without too many words) the analogy between today's world, which consists of dedicated resources per application, to the cloud world, which consists of a virtualized pool of resources. As can be seen in this diagram, Applications morphs into Software-as-a-Service, Middleware morphs into Platform-as-a-Service and Operating-System, Hardware and Network morphs into Infrastructure-as-a-Service.