rdbms as backend

[Leif Pedersen]

Hm, lemmie step back a sec. So as I understand, you currently get 30k
frontend requests per sec, and with Varnish to cache results, you have
about 7k backend requests per sec. Does this line up now?

Seems to me that if you're provisioning the middleware for 7k rqs instead
of 30k rqs, the problem is much easier to solve. It may require a few
machines, but it sounds like your DB costs are so high that saving you 76%
on database traffic would be an easy budget to meet. You've done FAPWS3 on
one machine at 3k rps? How about simply running that solution on 3 machines
plus a spare or two, which should provision it for about 9k rps? One nice
thing about this approach is that it's usually far easier to add (and fail
over) HTTP nodes and database clients than to add database servers.

I'm pragmatic in this. If the middleware costs a lot more than a custom
vmod to connect to the DBs, then I'd most likely do that. My skepticism is
just that it doesn't seem likely. So I won't answer your question ("you
would not connect to DB directly either?") in the absolute affirmative.
However, with an experienced guess and only a little information about your
problem space, that is my inclination, yes. And I wouldn't build the vmod
for purity or fun -- it sounds to me like a daunting gnarly thing with lots
of maintainability issues. But don't let me tell you not to, if you really
believe in the cause. :)

With deference to the authors, I'd be a bit astonished to see such a vmod
in Varnish's distribution. But if it's worth it in comparison to a
middleware solution (be it Python, node.js, C++, or whatever), the results
would certainly be interesting as a third-party vmod if you don't mind
sharing.

- Leif


-- 

As implied by email protocols, the information in this message is
not confidential.  Any middle-man or recipient may inspect, modify,
copy, forward, reply to, delete, or filter email for any purpose unless
said parties are otherwise obligated.  As the sender, I acknowledge that
I have a lower expectation of the control and privacy of this message
than I would a post-card.  Further, nothing in this message is
legally binding without cryptographic evidence of its integrity.

http://bilbo.hobbiton.org/wiki/Eat_My_Sig
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-dev/attachments/20130801/cfce71d6/attachment.html>