Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove the bluebird dependency from everywhere 🔥 #14882

Open
ErisDS opened this issue May 23, 2022 · 12 comments
Open

Remove the bluebird dependency from everywhere 🔥 #14882

ErisDS opened this issue May 23, 2022 · 12 comments
Labels
Hacktoberfest Issues suitable for hacktoberfest participants help wanted [triage] Ideal issues for contributors to help with pinned [triage] Ignored by stalebot

Comments

@ErisDS
Copy link
Member

ErisDS commented May 23, 2022

Now that there are native promises in Node.js and with async/await, bluebird really is a blast from the past. However, it's embedded pretty hard across our dependencies and some of our code uses catch predicates, which means we have to be careful about the order in which we rip bluebird out, otherwise we could break our error handling in weird and unexpected ways.

Here's some notes on what we need to do, most importantly starting with finding everywhere we depend on catch predicates:

Native:

  • Promise.all
  • Promise.then
  • Promise.reject
  • Promise.resolve

Bluebird

  • catch predicates -> replace with normal if statements
  • Promise.promisify -> require('util').promisify
  • Promise.props -> Promise.all example
  • Promise.join -> Promise.all
  • Promise.each -> Promise.all + array.map
  • Promise.map -> Promise.all + array.map
  • Promise.filter -> for and async/await
  • Promise.mapSeries -> use our sequence util (which needs an implementation of reduce, but is our only usage of it)
  • Promise.delay -> promisify getTimeout? Or get rid?

Refactor Approach

This refactor needs to be done in two phases:

  1. ✅ Remove all the catch predicates

💡 The best way to find catch predicates is with this regex: /\.catch\([^)]+,/.

  1. Remove bluebird from everywhere else:
  • Ghost monorepo - bluebird is required 117 times 😱
  • framework - bluebird required 5 times, but these are some of the most critical usages
  • SDK - just one usage, easy win!
  • gscan - 3 usages
  • knex-migrator - 3 usages

Why did we have to do catch predicates first?

Catch predicates result in error handling that assume the Promise they are handling is a Bluebird promise. If that isn't the case, Ghost will error, and quite possibly crash.

Our code coverage isn't quite good enough to be certain that all the unhappy paths are tested and we'll find these, and it's too much cognitive load for anyone to figure out if removing Bluebird is safe whilst there are still catch predicates in the codebase.


How to work on this issue

The slow and steady approach to refactoring is the only thing that really works. Branches should be very short lived (<1 day if possible). Therefore, please do not attempt to do this entire refactor in a single PR. It is not possible for us to review!

Instead, please find a single file with many references, or a small subfolder of the codebase with a handful of usages, make the changes, and submit a PR. Then repeat.

Please follow our contribution guidelines as closely as you can, particularly being sure to reference this issue in your commit. Refactor commits should not use emojis as they are not user-facing changes.

Note: This refactor has almost no user facing impact and (hopefully) no tests should break or change.

A reference refactor commit can be found here: 10aad8d

Thank you 🙏 ❤️ 🙏

@ErisDS ErisDS added the help wanted [triage] Ideal issues for contributors to help with label May 23, 2022
@Emmanuel-Melon
Copy link
Contributor

I can work with this, I had a similar task at my previous workplace 😁

Navarjun pushed a commit to Navarjun/Ghost that referenced this issue May 28, 2022
Issue TryGhost#14882
- Bluebird promise was being used for getting cached image sizes using .props method. It has been removed to be replaced by native promise using Promise.all
@ErisDS
Copy link
Member Author

ErisDS commented Jun 3, 2022

I've updated the OP here to be clearer that we need to remove catch predicates first, and why as there have been a couple of PRs that missed that detail.

@Emmanuel-Melon
Copy link
Contributor

I've updated the OP here to be clearer that we need to remove catch predicates first, and why as there have been a couple of PRs that missed that detail.

Apologies for the delay, I've been away for a bit. Resuming working on this task today.

ErisDS pushed a commit that referenced this issue Aug 1, 2022
refs: #14882

* refactored security.password to use native bcrypt promises
* refactored security.string to use more modern es features
ErisDS pushed a commit to Dave3of5/Ghost that referenced this issue Aug 5, 2022
ErisDS pushed a commit that referenced this issue Aug 5, 2022
refs #14882

- catch predicates make removing Bluebird from other parts of the code risky.
@ErisDS ErisDS added the Hacktoberfest Issues suitable for hacktoberfest participants label Aug 10, 2022
ErisDS added a commit to ErisDS/Ghost that referenced this issue Aug 24, 2022
refs: TryGhost#14882

- I found a common pattern where catch predicates were being used to catch non-existent models in destroy methods, and sometimes elsewhere in the API endpoints
- The use of predicates is deprecated, and we're working to remove them from everywhere, so that we can remove bluebird
- In order to still handle these errors correctly, we needed a small change to mw-error-handler so that it can detect EmptyResponse errors from bookshelf, as well as 404s
Note: there is a small change as a result of this - the context on these errors now says "Resource not found" instead of "{ModelName} not found".
- I think this is acceptable for now, as we will be reviewing these errors in more depth later. It's quite easy to make changes, we just have to decide what with proper design input
ErisDS added a commit to ErisDS/Ghost that referenced this issue Aug 24, 2022
refs: TryGhost#14882

- The use of predicates is deprecated, and we're working to remove them from everywhere, so that we can remove bluebird
- This should be the final piece of the puzzle in terms of predicates, from here we can start removing bluebird without concern that a predicate somewhere will explode
- Note: some of this code is poorly tested, but the refactors are very straightforward and minimal
ErisDS added a commit that referenced this issue Aug 24, 2022
refs: #14882

- I found a common pattern where catch predicates were being used to catch non-existent models in destroy methods, and sometimes elsewhere in the API endpoints
- The use of predicates is deprecated, and we're working to remove them from everywhere, so that we can remove bluebird
- In order to still handle these errors correctly, we needed a small change to mw-error-handler so that it can detect EmptyResponse errors from bookshelf, as well as 404s
Note: there is a small change as a result of this - the context on these errors now says "Resource not found" instead of "{ModelName} not found".
- I think this is acceptable for now, as we will be reviewing these errors in more depth later. It's quite easy to make changes, we just have to decide what with proper design input
ErisDS added a commit that referenced this issue Aug 24, 2022
refs: #14882

- The use of predicates is deprecated, and we're working to remove them from everywhere, so that we can remove bluebird
- This should be the final piece of the puzzle in terms of predicates, from here we can start removing bluebird without concern that a predicate somewhere will explode
- Note: some of this code is poorly tested, but the refactors are very straightforward and minimal
@ErisDS
Copy link
Member Author

ErisDS commented Aug 24, 2022

Hey all 👋 this issue was blocked on needing to remove the catch predicates first, so I've just merged a PR that removes all of them.

I've also updated the issue with the current state of things.

I'll go through the couple of blocked open PRs that remove some bits of bluebird asap. In the meantime, it'd be great to get some momentum on this and get rid of bluebird finally!

daniellockyer added a commit that referenced this issue Jul 11, 2023
refs #14882

- we're nuking our use of Bluebird in favor of native Promises
daniellockyer added a commit that referenced this issue Jul 11, 2023
refs #14882

- we're nuking our use of Bluebird in favor of native Promises, so this commit removes most of the usage we have in the core module
daniellockyer pushed a commit that referenced this issue Jul 11, 2023
refs #14882

-  removed Bluebird Promise.props from files in favor of native Promise.all
daniellockyer added a commit that referenced this issue Jul 11, 2023
refs #14882

- this is no longer used
daniellockyer added a commit that referenced this issue Jul 11, 2023
refs #14882

- this is no longer used so we can remove it as we move towards removing
  Bluebird from the codebase
daniellockyer added a commit that referenced this issue Jul 11, 2023
refs #14882

- this is no longer used so we can remove it as we move towards removing
  Bluebird from the codebase
chairulakmal added a commit to chairulakmal/Ghost that referenced this issue Aug 26, 2023
refs TryGhost#14882
with this the Bluebird is no longer required in ghost/core
@chairulakmal
Copy link
Contributor

chairulakmal commented Aug 26, 2023

Hello everyone,

I'm a new contributor here! I've created pull request #17831 to remove Bluebird from the ghost/core codebase. I found that only the raw-knex module was still using Bluebird, so I refactored its code made a minimal change to avoid using Bluebird Promise.props. Additionally, I removed Bluebird from ghost/core's package.json since no other part of the codebase needs it.

It's worth noting that the changes to the raw-knex plugin might not be fully covered by the current tests, as it seems only core/server/services/url/Resources uses raw-knex. I didn't realize there's a regression test specifically for core and my changes result in error. Will fix this today or tomorrow.

I ran yarn test:all for core and it yielded similar results to those of the current main branch. Your feedback would be greatly appreciated – looking forward to your thoughts on these updates!

Thanks,
kai

chairulakmal added a commit to chairulakmal/Ghost that referenced this issue Aug 28, 2023
refs TryGhost#14882
with this the Bluebird is no longer required in ghost/core
chairulakmal added a commit to chairulakmal/Ghost that referenced this issue Aug 29, 2023
daniellockyer pushed a commit that referenced this issue Aug 29, 2023
refs #14882

- this removes the last usage of Bluebird within Ghost and removes the dependency as it's no longer required within the codebase
daniellockyer pushed a commit to TryGhost/gscan that referenced this issue Aug 31, 2023
refs TryGhost/Ghost#14882

Co-authored-by: Princi Vershwal <princi.vershwal@Princis-MacBook-Pro.local>
daniellockyer pushed a commit to TryGhost/gscan that referenced this issue Aug 31, 2023
refs TryGhost/Ghost#14882

Co-authored-by: Princi Vershwal <princi.vershwal@Princis-MacBook-Pro.local>
daniellockyer added a commit to TryGhost/gscan that referenced this issue Aug 31, 2023
refs TryGhost/Ghost#14882

- this dependency is no longer needed because we're removed all
  functions that were supplied from it
markstos added a commit to markstos/express-hbs that referenced this issue Sep 3, 2023
Also, update eslintrc to support arrow notation.

Ref: TryGhost/Ghost#14882
markstos added a commit to markstos/express-hbs that referenced this issue Sep 3, 2023
Also, update eslintrc to support arrow notation.

Ref: TryGhost/Ghost#14882
@markstos
Copy link
Contributor

markstos commented Sep 3, 2023

I've submitted a PR to the nodemailer-mailgun-transport project to resolve this issue in that part of the dependency stack: orliesaurus/nodemailer-mailgun-transport#126

@markstos
Copy link
Contributor

markstos commented Sep 3, 2023

I've reviewed the status of this, and after the open PRs linked above are merged, it appears that perhaps than can be closed after lerna is used for final new package releases that contain the fix and and dependencies on those packages are updated one more time.

  • 🚧 Ghost monorepo - a yarn.lock mention remains which should get resolved per above.
  • ✅ framework
  • ✅ SDK
  • ✅ gscan
  • ✅ knex-migrator

daniellockyer pushed a commit to TryGhost/express-hbs that referenced this issue Sep 5, 2023
refs TryGhost/Ghost#14882

- this switches out the Bluebird specific code for a native variant so we can remove the dependency
@peterzimon peterzimon unpinned this issue Sep 26, 2023
@07tAnYa
Copy link

07tAnYa commented Oct 12, 2023

kindly assign this to me , I can work on this

@markstos
Copy link
Contributor

@07tAnYa Per my comment above, there is really nothing left to do. What change are you planning to make?

daniellockyer added a commit to TryGhost/knex-migrator that referenced this issue Dec 4, 2023
refs TryGhost/Ghost#14882

Co-authored-by: Daniel Lockyer <hi@daniellockyer.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Hacktoberfest Issues suitable for hacktoberfest participants help wanted [triage] Ideal issues for contributors to help with pinned [triage] Ignored by stalebot
Projects
None yet
Development

No branches or pull requests

8 participants