Previously new uncached requests for Open graph share-card images were generated in Chrome, sometimes taking up to 5 seconds to generate. (They were then stored in cache for 12 hrs).
Because we're now using Satori powered og images, they render almost immediately and we don't have to own the problem of having Chrome deployed within the 50mb lambda limitation. I'm pretty happy about that.
We're currently using Heroku for app hosting, while evaluating fly.io, render.com and railway.app. All three have had exceptional reviews from other customers and differ slightly on their service offerings and setups. All seem like viable alternatives so far!
I’d be really interested to know how your research goes. We are running a large production scale app on heroku. And that we have no DevOps or Infra staff is awesome. But heroku has been a huge let down lately.
Most of the reviews we have seen of the competitors are all hobby level. And last time we check some of this competitors we found their security posture was not the level we would require.
So we had to simple rule them out and either stay with Heroku or move to a big 3.
If you’re keen to share - let me know and I’ll send you my details.
Craig here from Crunchy Data. I think you're speaking to the app side of things on hobby level. On the database side of things our security posture for Crunchy Bridge I'd say is stronger than the Heroku one. By default we isolate all databases in a VPC, everything is purely single tenant where as Heroku Postgres at least when I was there had multiple forms of multi-tenancy which when doing multi-tenancy in Postgres can have risks[1]-this applies even to the major 3 cloud providers. Our team is essentially the original Heroku Postgres team so we've built with security but also user experience for Postgres in mind since day one.
Now I assume you were speaking to the 3 mentioned, render, railway, fly in terms of hobby level. All three are fairly young relative to Heroku's age, but Fly did recently get their SOC2 and the team really took it to heart and invested in it so I'd put some stock in that. I can't speak definitively to the others, but do know all three can be solid for production apps. If you've got HIPAA or other specific requirements I'd encourage a conversation with them.
Thanks Craig. I took at look at Crunchydata - however best I can tell unless we are in Enterprise Tier Heroku (or maybe not even then) we have to connect to Crunchydata via internet (with IP whitelist?) rather than through VPC peering or similar. Which is a limitation of Heroku rather than you. I assume with something like fly it could be done via VPC peering?
I just read fly had SOC2 type I recently. But I mean this hosting infra containing all our data and our customers data. People providing infra really need to take security extremely seriously and prove it.
Awesome what they are doing - just don’t feel like they are ready for primetime busines. We are a small startup (5k monthly on Heroku) but there is just no reasonable way we can tell our enterprise customers security teams are hosted on these guys and can vouch and vet their security.
This is the same path we're on. Migrated to Crunchy a month ago or so to remove the major migration risk and are using Render to host an auxiliary service while our core application remains on Heroku. Haven't yet done any non-toy deployments on Fly.io or Railway but I very much like Render's Blueprints and environment groups.
For what it's worth: Crunchydata appears to have been using Heroku DNS in some way. We have a Heroku app with Crunchy databases, and our Crunchy dbs became inaccessible shortly before the app did.
Craig here from Crunchy, we do run some small pieces on Heroku, but during the DNS outage saw no interruption of databases and they all seemed to be available and up. Please do feel free to drop us a note and we'll happily investigate if something did occur there.
I've left significant levels of details about pagespeed on the lighthouse issue tracker over the last few months. It's really frustrating to see the lighthouse team being blamed for something that isn't operationally even theirs… but that's a big company for you.
The biggest contributing factor to unstable pagespeed scores is the power of the machines that do the tests. Last time I checked the scores / bench mark scores for pagespeed were all over the place. … All the same, it's really important to understand how it all works. Hopefully the post had something for you?
Lighthouse definitely has some improvements to make for the audit suggestions that offer advice to improve the score. I think people get a little bit frustrated/blinded by the advice when there's nothing that can be done about third parties (I know I do!).
Our product Calibre (mentioned in the bottom of the post) has a n adblocking/third party blocking feature that allows you to compare your pages with or without third party scripts gunking up performance metrics (here's the release post: https://calibreapp.com/blog/release-notes-apr-2019/).
We've seen really good success in separating the two because it sends a clear signal of what a site WOULD score without ads. This is good because it makes it clear to PMs, management and decision makers that tracking tools incur a really visible cost.
You realise that calibre is also the name of a very well known ebook management app that's been around forever? I thought this was a post from them, initially. I'd imagine your website would get a lot of bounces due to this.
… but to answer your question better. It sucks that Google's own services don't live up to the performance expectations of tools made in other parts of Google.
I imagine the various teams at Google see each other as competition more than we might realize. How many chat-like applications are there? How about the Java/Kotlin Android teams versus the Flutter team? All of the GPL die-hards looking at the non-GPL (MIT, BSD, and Apache2) Fuchsia project wondering just how committed Google is to the Linux kernel.
if google could gain a market advantage similar to the one they got from linux (for free) they would do so immediately no matter the "cost" to linux/OSS/the-world-at-large
Why would someone want to pay you for your service, as opposed to using any of the above? Do you support browsers other than Chrome? Do you support a wide variety of simulated hardware? Do you support a wide variety of testing locations? Do you support any page speed test profile other than Lighthouse and Google Page Speed? Do you support running your software in a Docker container of my own choice?
Because I can get all those things via one or more of the competitors in this space, and I'm not seeing any compelling reason to even look at Calibre.
Depends on your use case - I recommend Calibre to some clients, SpeedCurve to others and self hosted WebPageTest or Sitespeed.io to others.
One of the key deciding factors is do you want to run the service yourself e.g. VMs, containers, in multiple regions or do you want to make that someone else's problem and get on with running your business - most of my clients are in the later camp
Both Calibre and SpeedCurve have clean UI's that allow performance to be tracked over time, from multiple locations and integration with other services via web hooks, APIs etc.
GTMetrix is old and outdated IMV, - it still heavily relies on YSlow rules which are well past their sell-by-date
WebPageTest is my favourite (SpeedCurve uses it under the hood) but it's got no ability to track over time, SiteSpee.io is really nice too but is self-hosted.
With the exception of some real devices in Dulles, all of the products rely on emulation for mobile, and Safari is a real gap for them all too
Without being super familiar with the tech that any of these companies are running —
The biggest thing that sets Calibre apart is that its entirely built using the Chrome developer tools APIs. I like to think about it as someone on your team who opened devtools after every deploy, and kept a neat spreadsheet of all the changes that had occurred. In short, all the data is ready and available for the taking.
This also has the added benefit that I have the power of the Chrome browser to play with. Projects like Google's Lighthouse (https://developers.google.com/web/tools/lighthouse/ — Which I'm currently integrating into Calibre!) have meant that a lot more powerful tools/metrics/resources/etc are coming along strongly, and this will only help make Calibre more powerful too.
Calibre is firmly focused on what your users actually experience, and helping you get to the bottom of 'why' quickly and efficiently. So while technically focused tools are imperative for any team, many of them aren't focused on the most important people in the equation.
Yeah… that animation isn't what it should be for a performance product—luckily, after this (and the lighthouse integration I'm working on right now), I'll have all-new marketing too!
Hi HN! I'm Ben, the solo founder of Calibre Analytics: https://calibreapp.com. Calibre monitors websites and applications in a clean-room testing environment from all over the world. Up until January Calibre has been an "inbetween other things" / evenings / holidays kind of project. (Only, oops—I did that for four years.).
Last year I knuckled down to get into the position where I could work on it full time. I've spent the last 8 weeks working on "Test profiles" (The launch post is here: https://medium.com/get-calibre/announcing-test-profiles-and-...) — It allows you to emulate hardware devices (cpu speed, user-agents and screen dimensions), as well as limit the bandwidth (latency, download and upload speeds), and set cookies (you could use this to log into your app, test a staff-shipped feature, or maybe turn off advertising and see the performance impact.)
I think I've finally got the basis of a really powerful tool, and I'm keen to share it with… everyone! I do everything myself, so if you've any questions, I'm here to answer them!
I have been using http://www.webpagetest.org/ for similar investigations. csn you tell me what you see as advantages of your tool over webpagetest.org ?
Web page test is a succhhh a great tool, and its really great for one off tests. Calibre will automatically hit your site daily (or, when you ping the snapshot api), and you'll have a full history of what happened and when.
With Calibre the aim is to make it a part of your toolchain (read: integrations) — so it should be visible to your entire team, or build system, or slack channel.
The App itself is boring old rails, with postgres (get me started on how much I like postgres and I will literally bore you to tears.) The original views were all haml and sass delivered via the asset pipeline. I sprinkled in angular here or there as the application grew.
Today, I'm slowly tearing away the angular. I've moved to webpack for JavaScript, and React for all new UIs. I've been really enjoying both the speed of development, and how solid everything feels.
Over time, I'm writing less and less ActiveRecord, and more and more sequel… still learning, but getting better.
The app is run on Heroku, and its been A++ would recommend to everyone.
The Calibre agent is node and websockets talking to Chrome®, theres other bits of ruby, and even some golang now.
I use terraform and ansible (although, the ansible may as well have been bash scripts) through a buildkite (https://buildkite.com — the best fucking CI service on the planet) pipeline. AMIs are built, shipped to different regions, then based on some config are rolled out.
Papertrail for logs - omg, logging! Essential. I run thousands of tests per day (I don't know this number off the top of my head, but its a lot)… and the nature of the internet is THERE IS ALWAYS FAILURE. Having logging at my fingertips has always made my job building Calibre easier. I think I pay $20 per month right now, and its the most important thing I pay for.
--
Ok. Whew. Could probably keep going, but I'm not sure how much you wanna know. Happy to answer specific things though!
I do everything. No contractors. Although @fox has been recently helping me with my writing (her native language is Polish, and she writes waayyy better than me - and most others).
Previously new uncached requests for Open graph share-card images were generated in Chrome, sometimes taking up to 5 seconds to generate. (They were then stored in cache for 12 hrs).
Because we're now using Satori powered og images, they render almost immediately and we don't have to own the problem of having Chrome deployed within the 50mb lambda limitation. I'm pretty happy about that.