Discourse Scaling and Storage

So now that this is up and running, we need to decide if we actually like it enough to keep. And if we DO want to keep it, we need to have a plan for what happens if we exceed the current limits for storage and/or processing speed.

Right now this thing is running off of a virtual machine hosted on Digital Ocean (www.digitalocean.com). It’s currently $6/m to host. Discourse is free, but to host it is not. Digital Ocean used to have a free tier but that’s gone away, so, $6/m.

That $6 gets us a virtual machine with 1gb of RAM and 25gb of storage space. We might need to upgrade to 2gb RAM/50gb storage eventually, depending on how much traffic this thing gets. We’re probably fine on the $6 tier for quite awhile, but it is something we need to consider. The current forum doesn’t cost anything since it’s all rolled up into hosting costs for everything else on our .com site.

We aren’t tied to any of this. The VM can easily be moved someplace else (we can even self host if we want but I don’t like that option) but I’m not sure we really want to deal with that. If we don’t like Discourse or don’t want to pay anything, we can just shut the VM off and I’ll pay the $6 and we can move on.

I don’t really have a point to all this. I just want to put it out there that the current forum is “free” is this one won’t always be so and we’ll need to be aware that at some point this thing might start costing us $100 - $200 a year.

2 Likes

I think a cost is to be expected at this point. My only concern is how long will this product exist, and does it stand alone if the company goes away.

Basically how long before we are in the same boat again? This is probably unanswerable, of course, but just something I have wondered.

2 Likes

Excellent question!

What is Discourse? | Discourse - Civilized Discussion has some info, but the gist of it is that Discourse is open source software, so there’s not a “company” that maintains it. This means that as long as there are people using it, it will probably stay up to date. The current maintainers say that they are committed to supporting for at least the next 10 years.

Obviously nothing is guaranteed, but I think we should be ok for quite some time.

2 Likes

I think $100-200/year is cheap at the price. But we need a lot more input before we switch over completely. And even if we don’t go with this, you shouldn’t have to pay anything. Any expense should be reimbursable by the club.

2 Likes

Agreed considering our old software is free but falling apart to be honest ,the rate does not seem bad.

1 Like

Regarding space requirements, are the posted images apart of the storage space used? If it is just text, then 25-50gb will last forever. If images are apart of that space, then it will go away very quickly indeed. What about backing up the forum? will the storage space be reclaimed if it gets too full? Meaning that when it gets full, older items will get either backed up or roll off and be deleted.

1 Like

The original, “current” forum has a total database size of 28mb. That’s pretty amazing since it goes back at least a decade, but it is all text posts. This one will definitely fill up a bit quicker since images/uploads are allowed, but it will remain to be seen how long it’ll take. Backups are contained within the VM, so disk space in general is something we’ll have to keep an eye on.

Of note though - Discourse themselves recommends not just paying for more storage and instead setting up an S3 bucket and offload file uploads to there. S3 is super cheap so that’s probably the best thing, assuming we don’t need to also upgrade the server RAM at the same time.

1 Like

S3 is a good choice for images, not really needed for text. If the majority of images are links to external sites, that’ll reduce need for storage even more. But I bet due to ease of posting images, lots of members will be uploading from personal devices. I imagine that the images will be accessed each time the page is loaded by a user. Whether stored locally or in an S3 bucket.

Would that be set as an automatic upload to the S3 bucket? Or would you try to save the cost of frequent access and back up the images at a later date?

1 Like

More good questions. We can begin using an S3 bucket whenever. It’s not something we need to decide until we run out of space. S3 would hold all the uploads and backups.

I missed your last question about backups. Right now it automatically runs a backup every week. The default is every day but that seems really excessive for our needs. I do need to check on post expiration. The old forum never deletes anything and everyone keeps talking about old stuff we don’t want to lose. I expect we won’t want anything on this to expire either.

1 Like

The old forum does have an automatic purge that can be turned on. I think it is on for some areas like the swap and shop. I think it is set for one year after no post are made to the topic. It’s nice having history but not really necessary. The astrophotography will end up taking up most of the space. I clean posts out that are not relevant so it’s not so cluttered. Like posts with dead links or old information.

The current forum has been up for 18 years this month.

2 Likes

[quote=“DaveNH6SM, post:10, topic:41, full:true”]…
The current forum has been up for 18 years this month.
[/quote]

That is quite impressive. It served us well. Now we bid it a fond adieu.

2 Likes