Message buses, agents and web applications
Par Sébastien Pierre, mardi 22 septembre 2009 à 11:39 :: General :: #103 :: rss
I've been quite busy working on dataviz-related project, but also web-related projects which are making it to production, and on that topic I has some thoughts I'd like to share 
Something like 8 years ago I was still studying at the engineering school, working on the theory and practice of agent systems. Agents were a much-hyped term at that time, but they appeared to me as an evolution of OOP towards more scalable and more maintainable architectures.
The essence of agents (or actors, to a lesser degree) is autonomy (concurrency) and communication (message-passing) -- concepts which people having played with any serious massively concurrent, distributed platform (like Erlang) will be familiar with. In practice, agents are really just "objects with an inbox" with the ability to have access to an execution thread to schedule tasks.
What makes agents more interesting though is that using them will impact the way you architecture your application. Instead of having a monolithic application that process everything "vertically", you have an application composed of "horizontal" services delivered by agents.
To put this in context, you might have a Web application that uses XMPP, SMTP and database services which are available in the application environment -- the XMPP, SMTP and database services might be offered by one (or more) agents. You then don't need to have a heavyweight Web application importing a sh**load of dependencies, but simply benefit from the high-level by using a simple communication protocol.
There is nothing new here, the concepts were already there in CORBA 15 years ago, but with today's focus on "cloud computing" the agent paradigm seems to me as more interesting than ever -- because we now have a context to put it to good use. People using GNOME may know of D-BUS, which provides a similar architecture to enable easy "exchange of sevices" between applications.
So even in the modest web applications I'm working on, I already see the advantages of using an agent-based approach to designing the apps. However, I also feel some pain due to the lack of a consistent "agent platform" -- I make do with assembling selected components to form a "poor's man agent platform".
Here are the pieces you'd need to make your own:
- a message bus: something that allows inter-process (and ideally over-network) communication of messages. It's up to you to choose your message format, (I've personnaly chosen JSON with a standard type for messages) and your communication channel (spread or beanstalkd work quite well). I'm thinking about using D-BUS as well, as it (apparently) has a good support for managing your services.
- processes, threads or coroutines: agents need a way to poll their inbox and schedule arbitrary bits of code (either in reaction to messages, or in a proactive way). Implementing services with agents means that you'll use other lower-level services, and that communication will be made over the message bus. As a consequence you need to have an efficient way of parallelizing and scheduling the polling of inbox, polling of response status and execution of scheduled code. Nowadays having an event-based reactor and a simple promises/futures API is what seems to be works best -- but it's still difficult because of so-so language support. The alternative is to limit the use of the message bus and ensure that a request never fails... which requires a leap of faith

- administrative tools: because we're talking about a platform, you need things to ensure that your services are running, monitor the load, log problems, etc. Ideally, this could be build on the agent platform itself, by implementing watchdog services so that the necessary "bootstrapping tools" would be kept to a minimum. So far, I make do with crontabs, daemons and probably monit.
Anyway, I'm quite happy to see that the agent paradigm finds relevant application in the domain of web applications, even if there is still some work to be done until we can get a usable platform -- but it seems like most of the pieces are here !
Commentaires
1. Le mercredi 23 septembre 2009 à 17:01, par William Lachance
2. Le mardi 6 octobre 2009 à 04:46, par Pierre
3. Le samedi 24 juillet 2010 à 08:03, par mbt shoes on sale
4. Le mercredi 25 août 2010 à 00:11, par coach outlet
5. Le mercredi 25 août 2010 à 12:11, par research paper
6. Le jeudi 26 août 2010 à 00:30, par dresses
Ajouter un commentaire