[Admin-discuss] Dovecot
Hi all, At the suggestion of ryaner (thanks!), we've put a test copy of Dovecot IMAP server on deathray. This has a few advantages over courier: - It's faster (indexing, caching, stuff). - It lets me log in (courier refuses to accept that I exist, and shows nothing in the error logs). - It's website (http://www.dovecot.org/) says lots of nice things about it. I haven't added our SSL cert to it just yet, I'll do that in the next few minutes. It saves it's indexes locally on deathray in /var/indexes/$username, making it nice and fast. If anyone wants to try it out, just create a Secure IMAP (SSL) connection to deathray.redbrick.dcu.ie, and play around with it. -Andrew
On Sun, Aug 10, 2008 at 11:39:50PM +0100, Andrew Martin wrote:
Hi all,
At the suggestion of ryaner (thanks!), we've put a test copy of Dovecot IMAP server on deathray. This has a few advantages over courier:
If anyone wants to try it out, just create a Secure IMAP (SSL) connection to deathray.redbrick.dcu.ie, and play around with it.
Just tried it out there now, seems to be working fine. a. -- Andrew Harford System Administrator, DCU Networking Society Ordinary Member, Societies & Publications Committee You talk about vengeance. Is vengeance going to bring your son back to you? Or my boy to me? --Don Vito Corleone
On Sun, Aug 10, 2008 at 11:39:50PM +0100, Andrew Martin wrote:
Hi all,
At the suggestion of ryaner (thanks!), we've put a test copy of Dovecot IMAP server on deathray. This has a few advantages over courier:
- It's faster (indexing, caching, stuff). - It lets me log in (courier refuses to accept that I exist, and shows nothing in the error logs). - It's website (http://www.dovecot.org/) says lots of nice things about it.
I haven't added our SSL cert to it just yet, I'll do that in the next few minutes.
It saves it's indexes locally on deathray in /var/indexes/$username, making it nice and fast.
If anyone wants to try it out, just create a Secure IMAP (SSL) connection to deathray.redbrick.dcu.ie, and play around with it.
Few things to mention are that dovecot and courier uses different subscription lists and namespaces. http://wiki.dovecot.org/Migration/Courier details a good part of it and they have done a lot of work over the past while making the change easier. Basic steps would be to copy "courierimapsubscribed" to "subscriptions" for every one but have it changed to remove the inbox prefixes, then copy the uid files if you want. RB has enough cpu/memory so as not to notice a uid rebuild, even for the largest of users here. -- This is the signature Really??? Imagine that
On Mon, Aug 11, 2008 at 06:49:15AM +0100, Ryaner wrote:
On Sun, Aug 10, 2008 at 11:39:50PM +0100, Andrew Martin wrote:
Hi all,
At the suggestion of ryaner (thanks!), we've put a test copy of Dovecot IMAP server on deathray. This has a few advantages over courier:
- It's faster (indexing, caching, stuff). - It lets me log in (courier refuses to accept that I exist, and shows nothing in the error logs). - It's website (http://www.dovecot.org/) says lots of nice things about it.
I haven't added our SSL cert to it just yet, I'll do that in the next few minutes.
It saves it's indexes locally on deathray in /var/indexes/$username, making it nice and fast.
If anyone wants to try it out, just create a Secure IMAP (SSL) connection to deathray.redbrick.dcu.ie, and play around with it.
Few things to mention are that dovecot and courier uses different subscription lists and namespaces. http://wiki.dovecot.org/Migration/Courier details a good part of it and they have done a lot of work over the past while making the change easier. Basic steps would be to copy "courierimapsubscribed" to "subscriptions" for every one but have it changed to remove the inbox prefixes, then copy the uid files if you want. RB has enough cpu/memory so as not to notice a uid rebuild, even for the largest of users here.
That looks good.. I like the PostLoginScripting option to convert people's UIDs the first time they log in.. we might want to add a couple of conditionals to that to make sure that they don't run it twice, but the idea should work. About the indexes - The way I'm doing it now is using a perl script to create a bunch of directories belonging to users under /var/indexes. This means that we don't have to worry about NFS limitations on dovecot's indexing. (It doesn't like saving it's indexes on NFS volumes too much, it has to apply workarounds which reduce performance). Because deathray isn't accessable to users, we can safely assume that once WWW is moved back to murphy, there'll be no way for them to write $random_crap to their index directory (essentially giving them extra local storage). We're not being stingy or anything, but that particular disk space is valuable. Anyway, this obviously won't work on minerva. So, do we want to come up with an alternate (NFS safe, so less fast) solution for minerva, or just plan on moving production IMAP back to deathray? Did any of that even make sense to anyone? Congratulations if it did, I'm quite tired writing this :) -Andrew
On Thu, Aug 14, 2008 at 11:03:23PM +0100, Andrew Martin wrote:
On Mon, Aug 11, 2008 at 06:49:15AM +0100, Ryaner wrote:
On Sun, Aug 10, 2008 at 11:39:50PM +0100, Andrew Martin wrote:
Hi all,
At the suggestion of ryaner (thanks!), we've put a test copy of Dovecot IMAP server on deathray. This has a few advantages over courier:
- It's faster (indexing, caching, stuff). - It lets me log in (courier refuses to accept that I exist, and shows nothing in the error logs). - It's website (http://www.dovecot.org/) says lots of nice things about it.
I haven't added our SSL cert to it just yet, I'll do that in the next few minutes.
It saves it's indexes locally on deathray in /var/indexes/$username, making it nice and fast.
If anyone wants to try it out, just create a Secure IMAP (SSL) connection to deathray.redbrick.dcu.ie, and play around with it.
Few things to mention are that dovecot and courier uses different subscription lists and namespaces. http://wiki.dovecot.org/Migration/Courier details a good part of it and they have done a lot of work over the past while making the change easier. Basic steps would be to copy "courierimapsubscribed" to "subscriptions" for every one but have it changed to remove the inbox prefixes, then copy the uid files if you want. RB has enough cpu/memory so as not to notice a uid rebuild, even for the largest of users here.
That looks good.. I like the PostLoginScripting option to convert people's UIDs the first time they log in.. we might want to add a couple of conditionals to that to make sure that they don't run it twice, but the idea should work.
About the indexes - The way I'm doing it now is using a perl script to create a bunch of directories belonging to users under /var/indexes.
This means that we don't have to worry about NFS limitations on dovecot's indexing. (It doesn't like saving it's indexes on NFS volumes too much, it has to apply workarounds which reduce performance).
Because deathray isn't accessable to users, we can safely assume that once WWW is moved back to murphy, there'll be no way for them to write $random_crap to their index directory (essentially giving them extra local storage). Procmail? We're not being stingy or anything, but that particular disk space is valuable. Anyway, this obviously won't work on minerva.
So, do we want to come up with an alternate (NFS safe, so less fast) solution for minerva, or just plan on moving production IMAP back to deathray?
Did any of that even make sense to anyone? Congratulations if it did, I'm quite tired writing this :)
-Andrew
_______________________________________________ Admin-discuss mailing list Admin-discuss@lists.redbrick.dcu.ie http://lists.redbrick.dcu.ie/mailman/listinfo/admin-discuss
-- --
On Fri, Aug 15, 2008 at 12:32:55AM +0100, Cian Brennan wrote:
On Thu, Aug 14, 2008 at 11:03:23PM +0100, Andrew Martin wrote:
On Mon, Aug 11, 2008 at 06:49:15AM +0100, Ryaner wrote:
On Sun, Aug 10, 2008 at 11:39:50PM +0100, Andrew Martin wrote:
Hi all,
At the suggestion of ryaner (thanks!), we've put a test copy of Dovecot IMAP server on deathray. This has a few advantages over courier:
- It's faster (indexing, caching, stuff). - It lets me log in (courier refuses to accept that I exist, and shows nothing in the error logs). - It's website (http://www.dovecot.org/) says lots of nice things about it.
I haven't added our SSL cert to it just yet, I'll do that in the next few minutes.
It saves it's indexes locally on deathray in /var/indexes/$username, making it nice and fast.
If anyone wants to try it out, just create a Secure IMAP (SSL) connection to deathray.redbrick.dcu.ie, and play around with it.
Few things to mention are that dovecot and courier uses different subscription lists and namespaces. http://wiki.dovecot.org/Migration/Courier details a good part of it and they have done a lot of work over the past while making the change easier. Basic steps would be to copy "courierimapsubscribed" to "subscriptions" for every one but have it changed to remove the inbox prefixes, then copy the uid files if you want. RB has enough cpu/memory so as not to notice a uid rebuild, even for the largest of users here.
That looks good.. I like the PostLoginScripting option to convert people's UIDs the first time they log in.. we might want to add a couple of conditionals to that to make sure that they don't run it twice, but the idea should work.
About the indexes - The way I'm doing it now is using a perl script to create a bunch of directories belonging to users under /var/indexes.
This means that we don't have to worry about NFS limitations on dovecot's indexing. (It doesn't like saving it's indexes on NFS volumes too much, it has to apply workarounds which reduce performance).
Because deathray isn't accessable to users, we can safely assume that once WWW is moved back to murphy, there'll be no way for them to write $random_crap to their index directory (essentially giving them extra local storage). Procmail? We're not being stingy or anything, but that particular disk space is valuable. Anyway, this obviously won't work on minerva.
Why won't they be able to write to the index directory?
So, do we want to come up with an alternate (NFS safe, so less fast) solution for minerva, or just plan on moving production IMAP back to deathray?
Did any of that even make sense to anyone? Congratulations if it did, I'm quite tired writing this :)
-Andrew
_______________________________________________ Admin-discuss mailing list Admin-discuss@lists.redbrick.dcu.ie http://lists.redbrick.dcu.ie/mailman/listinfo/admin-discuss
--
--
_______________________________________________ Admin-discuss mailing list Admin-discuss@lists.redbrick.dcu.ie http://lists.redbrick.dcu.ie/mailman/listinfo/admin-discuss
-- Andrew Harford System Administrator, DCU Networking Society Ordinary Member, Societies & Publications Committee Ultimately, we're all dead men. Sadly, we cannot choose how, but what we can decide is how we meet that end, in order that we are remembered as men. --Proximo (Gladiator)
On Fri, Aug 15, 2008 at 09:01:13AM +0100, Andrew Harford wrote:
On Fri, Aug 15, 2008 at 12:32:55AM +0100, Cian Brennan wrote:
On Thu, Aug 14, 2008 at 11:03:23PM +0100, Andrew Martin wrote:
On Mon, Aug 11, 2008 at 06:49:15AM +0100, Ryaner wrote:
On Sun, Aug 10, 2008 at 11:39:50PM +0100, Andrew Martin wrote:
Hi all,
At the suggestion of ryaner (thanks!), we've put a test copy of Dovecot IMAP server on deathray. This has a few advantages over courier:
- It's faster (indexing, caching, stuff). - It lets me log in (courier refuses to accept that I exist, and shows nothing in the error logs). - It's website (http://www.dovecot.org/) says lots of nice things about it.
I haven't added our SSL cert to it just yet, I'll do that in the next few minutes.
It saves it's indexes locally on deathray in /var/indexes/$username, making it nice and fast.
If anyone wants to try it out, just create a Secure IMAP (SSL) connection to deathray.redbrick.dcu.ie, and play around with it.
Few things to mention are that dovecot and courier uses different subscription lists and namespaces. http://wiki.dovecot.org/Migration/Courier details a good part of it and they have done a lot of work over the past while making the change easier. Basic steps would be to copy "courierimapsubscribed" to "subscriptions" for every one but have it changed to remove the inbox prefixes, then copy the uid files if you want. RB has enough cpu/memory so as not to notice a uid rebuild, even for the largest of users here.
That looks good.. I like the PostLoginScripting option to convert people's UIDs the first time they log in.. we might want to add a couple of conditionals to that to make sure that they don't run it twice, but the idea should work.
About the indexes - The way I'm doing it now is using a perl script to create a bunch of directories belonging to users under /var/indexes.
This means that we don't have to worry about NFS limitations on dovecot's indexing. (It doesn't like saving it's indexes on NFS volumes too much, it has to apply workarounds which reduce performance).
Because deathray isn't accessable to users, we can safely assume that once WWW is moved back to murphy, there'll be no way for them to write $random_crap to their index directory (essentially giving them extra local storage). Procmail? We're not being stingy or anything, but that particular disk space is valuable. Anyway, this obviously won't work on minerva.
Why won't they be able to write to the index directory?
Sorry, should've been clearer. On deathray, I've created for each user a /var/indexes/$username directory. We don't *want* them to be able to write to this themselves, which at the moment isn't a problem. But, if we move back to minerva and use the same setup, then obviously they will be able to write to /var/indexes. (they can log into the machine). The assumption on deathray is that WWW will be moved back to murphy in the very near future. (WWW provides another way for them to write to their index directory). Cian's point about procmail is right though, that's executed on deathray, so could in theory be used to do whatever they want there. (thinking about it more, this is a security bug in general, users shouldn't be able to execute code on deathray).
So, do we want to come up with an alternate (NFS safe, so less fast) solution for minerva, or just plan on moving production IMAP back to deathray?
Did any of that even make sense to anyone? Congratulations if it did, I'm quite tired writing this :)
-Andrew
_______________________________________________ Admin-discuss mailing list Admin-discuss@lists.redbrick.dcu.ie http://lists.redbrick.dcu.ie/mailman/listinfo/admin-discuss
--
--
_______________________________________________ Admin-discuss mailing list Admin-discuss@lists.redbrick.dcu.ie http://lists.redbrick.dcu.ie/mailman/listinfo/admin-discuss
-- Andrew Harford System Administrator, DCU Networking Society Ordinary Member, Societies & Publications Committee
Ultimately, we're all dead men. Sadly, we cannot choose how, but what we can decide is how we meet that end, in order that we are remembered as men. --Proximo (Gladiator)
On Fri, Aug 15, 2008 at 09:38:17AM +0100, Andrew Martin wrote:
On Fri, Aug 15, 2008 at 09:01:13AM +0100, Andrew Harford wrote:
Why won't they be able to write to the index directory?
Sorry, should've been clearer.
On deathray, I've created for each user a /var/indexes/$username directory. We don't *want* them to be able to write to this themselves, which at the moment isn't a problem. But, if we move back to minerva and use the same setup, then obviously they will be able to write to /var/indexes. (they can log into the machine).
Is there a problem if someone does write to the index directory.
Cian's point about procmail is right though, that's executed on deathray, so could in theory be used to do whatever they want there. (thinking about it more, this is a security bug in general, users shouldn't be able to execute code on deathray).
I'm not sure there's a way around this if deathray is going to process mail. a. -- Andrew Harford System Administrator, DCU Networking Society Ordinary Member, Societies & Publications Committee Well, he's kind of had it in for me ever since I accidentally ran over his dog. Actually, replace "accidentally" with "repeatedly," and replace "dog" with "son." -- Lionel Hutz
On Fri, Aug 15, 2008 at 09:41:27AM +0100, Andrew Harford wrote:
On Fri, Aug 15, 2008 at 09:38:17AM +0100, Andrew Martin wrote:
On Fri, Aug 15, 2008 at 09:01:13AM +0100, Andrew Harford wrote:
Why won't they be able to write to the index directory?
Sorry, should've been clearer.
On deathray, I've created for each user a /var/indexes/$username directory. We don't *want* them to be able to write to this themselves, which at the moment isn't a problem. But, if we move back to minerva and use the same setup, then obviously they will be able to write to /var/indexes. (they can log into the machine).
Is there a problem if someone does write to the index directory.
Massive security problem? No. I'm just not sure we should be giving up local storage on deathray in an uncontrolled way. I may be wrong. (The procmail point still stands though, the idea of deathray is that users can't execute things there, after WWW goes).
Cian's point about procmail is right though, that's executed on deathray, so could in theory be used to do whatever they want there. (thinking about it more, this is a security bug in general, users shouldn't be able to execute code on deathray).
I'm not sure there's a way around this if deathray is going to process mail.
a.
-- Andrew Harford System Administrator, DCU Networking Society Ordinary Member, Societies & Publications Committee
Well, he's kind of had it in for me ever since I accidentally ran over his dog. Actually, replace "accidentally" with "repeatedly," and replace "dog" with "son." -- Lionel Hutz
On Fri, Aug 15, 2008 at 09:44:57AM +0100, Andrew Martin wrote:
Is there a problem if someone does write to the index directory.
Massive security problem? No.
I'm just not sure we should be giving up local storage on deathray in an uncontrolled way. I may be wrong.
(The procmail point still stands though, the idea of deathray is that users can't execute things there, after WWW goes).
Why not setup two mail servers? Have deathray as a store/forward server, and then one of the others to do the actual maildrop. That was their procmail can run on the other machine, you still have the benifits of two servers, and then deathray can still do all the imap logins. Sure you could even setup imap.rb on a virtual IP and have a heartbeat between the two so if one goes down, the ip moves. -- This is the signature Really??? Imagine that
On Fri, Aug 15, 2008 at 10:57:25AM +0100, Ryaner wrote:
On Fri, Aug 15, 2008 at 09:44:57AM +0100, Andrew Martin wrote:
Is there a problem if someone does write to the index directory.
Massive security problem? No.
I'm just not sure we should be giving up local storage on deathray in an uncontrolled way. I may be wrong.
(The procmail point still stands though, the idea of deathray is that users can't execute things there, after WWW goes).
Why not setup two mail servers? Have deathray as a store/forward server, and then one of the others to do the actual maildrop.
Is there any benifit to this, if deathray is just passing the mail elsewhere?
That was their procmail can run on the other machine, you still have the benifits of two servers, and then deathray can still do all the imap logins. Sure you could even setup imap.rb on a virtual IP and have a heartbeat between the two so if one goes down, the ip moves.
imap.rb already has a service ip :) -- Andrew Harford System Administrator, DCU Networking Society Ordinary Member, Societies & Publications Committee I spent my whole life trying not to be careless. Women and children can be careless. But not men. --Don Vito Corleone
On Fri, Aug 15, 2008 at 11:37:55AM +0100, Andrew Harford wrote:
On Fri, Aug 15, 2008 at 10:57:25AM +0100, Ryaner wrote:
On Fri, Aug 15, 2008 at 09:44:57AM +0100, Andrew Martin wrote:
Is there a problem if someone does write to the index directory.
Massive security problem? No.
I'm just not sure we should be giving up local storage on deathray in an uncontrolled way. I may be wrong.
(The procmail point still stands though, the idea of deathray is that users can't execute things there, after WWW goes).
Why not setup two mail servers? Have deathray as a store/forward server, and then one of the others to do the actual maildrop.
Is there any benifit to this, if deathray is just passing the mail elsewhere?
If you only have one mail server, and it goes down, mail goes no where or just sits in the other queue. With a second server doing store/forward, you can accept the mail into your own queue and then have it do agressive retries to mail is only delayed as long as the primary server is offline.
That was their procmail can run on the other machine, you still have the benifits of two servers, and then deathray can still do all the imap logins. Sure you could even setup imap.rb on a virtual IP and have a heartbeat between the two so if one goes down, the ip moves.
imap.rb already has a service ip :)
-- Andrew Harford System Administrator, DCU Networking Society Ordinary Member, Societies & Publications Committee
I spent my whole life trying not to be careless. Women and children can be careless. But not men. --Don Vito Corleone
-- This is the signature Really??? Imagine that
On Fri, Aug 15, 2008 at 11:59:17AM +0100, Ryaner wrote:
On Fri, Aug 15, 2008 at 11:37:55AM +0100, Andrew Harford wrote:
On Fri, Aug 15, 2008 at 10:57:25AM +0100, Ryaner wrote:
On Fri, Aug 15, 2008 at 09:44:57AM +0100, Andrew Martin wrote:
Is there a problem if someone does write to the index directory.
Massive security problem? No.
I'm just not sure we should be giving up local storage on deathray in an uncontrolled way. I may be wrong.
(The procmail point still stands though, the idea of deathray is that users can't execute things there, after WWW goes).
Why not setup two mail servers? Have deathray as a store/forward server, and then one of the others to do the actual maildrop.
Is there any benifit to this, if deathray is just passing the mail elsewhere?
If you only have one mail server, and it goes down, mail goes no where or just sits in the other queue. With a second server doing store/forward, you can accept the mail into your own queue and then have it do agressive retries to mail is only delayed as long as the primary server is offline.
My understanding of it is a bit vague, but from what I can gather... I kinda like this idea. Although HEAnet do store and forward for us (or DCU.. or somebody), this doesn't help us for inbound mail over IPv6 (as I understand it). Deathray + cynic in this sort of setup perhaps?
That was their procmail can run on the other machine, you still have the benifits of two servers, and then deathray can still do all the imap logins. Sure you could even setup imap.rb on a virtual IP and have a heartbeat between the two so if one goes down, the ip moves.
imap.rb already has a service ip :)
-- Andrew Harford System Administrator, DCU Networking Society Ordinary Member, Societies & Publications Committee
I spent my whole life trying not to be careless. Women and children can be careless. But not men. --Don Vito Corleone
-- This is the signature Really??? Imagine that
_______________________________________________ Admin-discuss mailing list Admin-discuss@lists.redbrick.dcu.ie http://lists.redbrick.dcu.ie/mailman/listinfo/admin-discuss
On Fri, Aug 15, 2008 at 12:07:36PM +0100, Andrew Martin wrote:
On Fri, Aug 15, 2008 at 11:59:17AM +0100, Ryaner wrote:
If you only have one mail server, and it goes down, mail goes no where or just sits in the other queue. With a second server doing store/forward, you can accept the mail into your own queue and then have it do agressive retries to mail is only delayed as long as the primary server is offline.
My understanding of it is a bit vague, but from what I can gather...
I kinda like this idea. Although HEAnet do store and forward for us (or DCU.. or somebody), this doesn't help us for inbound mail over IPv6 (as I understand it).
Deathray + cynic in this sort of setup perhaps?
That makes sense, send out going mail on these, and have them store the incoming mail until it can be delivered to carbon. -- Andrew Harford System Administrator, DCU Networking Society Ordinary Member, Societies & Publications Committee Sorry Meg. Daddy loves ya, but Daddy also loves Star Trek, and in all fairness, Star Trek was here first. --Peter Griffin
On Sun, Aug 10, 2008 at 11:39:50PM +0100, Andrew Martin wrote:
Hi all,
At the suggestion of ryaner (thanks!), we've put a test copy of Dovecot IMAP server on deathray. This has a few advantages over courier:
- It's faster (indexing, caching, stuff). - It lets me log in (courier refuses to accept that I exist, and shows nothing in the error logs). - It's website (http://www.dovecot.org/) says lots of nice things about it.
I haven't added our SSL cert to it just yet, I'll do that in the next few minutes.
Longest "next few minutes" ever, but I've added our SSL cert to it. I'd encourage people to try it out, find any problems with it (as ryaner said though, UIDs will be messed up, so your client may redownload everything). Best to find the quirks now :)
participants (4)
-
Andrew Harford -
Andrew Martin -
Cian Brennan -
Ryaner