Hacker News new | past | comments | ask | show | jobs | submit login
Billing systems are a nightmare for engineers (getlago.com)
777 points by Rafsark on May 18, 2022 | hide | past | favorite | 359 comments



I was working on a billing system for utility company and it was a nightmare but not for the reasons from the article.

Dates are no problem, everything is billed at the end of month, quarter or year.

Upgrades are downgrades are simplified, customer can legally change tariff only on predefined dates/events.

Usage - yes, but exampled in the article do not even scratch the complexity of this

Idempotency is not really needed, the billing process is not continuous, it is a batch job.

Cash collection is out of scope of billing software.

Taxing was clear. Yes, there were multiple billable items with different tax rates, but not too complex. All customers were local.

The nightmare part was:

- customers can have multiple consumption locations, location can have multiple meters and customer can request to split the bill on multiple invoices or combine multiple bills to single invoice as they please

- meters can be replaced at any time

- meter readings are available on dates totally independent from billing cycle. Most of consumption data are mere forecasts.

- when the actual consumption data is available, everything must be recalculated AND compensated (on a correction invoice showing the difference)

- actual consumption data can be wrong and may be corrected at a later date, even multiple times

- consumption points can be added or removed or moved to different customer at any time, but this information is only available after the fact

- the prices can change mid-billing cycle for some customers but the consumption data is not available with that granularity

- customer legal information (company name, address) can change mid-cycle


Thanks janci for sharing your experience!

Indeed, we only scratched the surface of the complexity of metered billing, we'll do a deep dive soon on this topic (it does deserve its own post).

I think for the'nightmares' you mention: - some might be specific to utility (not applicable to an online business), such as '- meter readings are available on dates totally independent from billing cycle'. - Some topics might be simpler for a utility co: tax for instance (you know where your users are, and they are limited to a geographic region)

- But some nightmares such as '- the prices can change mid-billing cycle for some customers but the consumption data is not available with that granularity' do really resonate for online businesses too, indeed

Thanks for sharing, great insights for our future post!


For online businesses as well, selling above a certain amount of revenue or product can trigger what's known as a nexus event in taxation wherein the business becomes liable for collecting sales or use tax. Now online businesses are on the hook for collecting sales tax in every jurisdiction above wildly varying threshold values. Should the business fail to collect the sales tax for selling from Montana to California for instance, then the business will be liable for sales tax not collected for products or services sold to California residents online even though the business might physically reside in Montana.


Is there an easy way to deal with this e.g. compliance as a service?

Sounds like a nightmare to deal with, of course this is only assuming it is enforced. If it is not well who cares, companies like Lyft/Uber view laws as a starting point in negotiations when expanding into new areas not as a brick wall of dogma.


> Is there an easy way to deal with this e.g. compliance as a service?

I think we call ourselves CPAs :). In all seriousness the 'solution' depends on what industry we're talking about. Short term rentals? Avalara. Software as a service? Various accounting firms and startups. Etc.


Montana does not have a sales tax.


Did the company use a relational database to deal with the billing and subsequent adjustments? I've been there: it can be more straightforward to use an event streaming approach that recalculates billing as new events arrive.


Yes, almost everything ended as temporal or bitemporal model. Pain to work with. Even for simple questions like "What is the customer #1234 company name?" the system needs to know when are you asking.


By "temporal model" do you mean relational tables of "customer X used Y amount of this resource at price Z from timestamp A to timestamp B" or ... ?


In field metering situations you can have the timeline of each meter, when it records data points, and then you have the different timeline of the backend service, when it actually _receives_ data points.

Bonus points if the meters can send data out of order.


I did a tender for aggregating events and emitting notifications from a fleet of transport vehicles that reported metrics over LTE.

When connected they would emit events as they happened, but when there was poor service or disconnection they would buffer them and send them as a burst once they reconnected... in reverse order.

I forget the finer details, but the service I was writing was supposed to notify of trucks entering and leaving geofences, inactivity etc. anything you can think of where the order of events is important.


Ah, thanks. OMFG and I thought our billing was hard. You're in my thoughts and prayers!


You just perfectly described what I’m currently working on. And it’s even more complex than you’ve explained (which I’m sure you’d well know haha)


I assume they are using something like DB2, which has very specific meaning when it comes to temporal or bitemporal tables.

https://www.ibm.com/docs/en/db2/10.5?topic=tables-bitemporal


Not DB2, but the meaning is the same.

In a time-series (or event) table you have one date - when the data point is valid (event occurred).

In a temporal table you have two dates - interval when the data is valid.

In a bitemporal table you have two intervals. First describes when the data is valid in real world, the second when does the system know about the real-world state.

I.e. if company changed address on 2022-05-01 but the system will not get that information until three day later (2022-05-03) you have two validity intervals.

Then you may ask - what was the company address on 2022-05-02? And what did the system think the address was on 2022-05-02? You will get two different answers. This is important if you need e.g. to make a correction to previously issued invoice. You need to know what data was used for original invoice and what data should have been used.


Designed and build the system meter readings and calculations for the Dutch market, all mysql, worked like a dream.


I have heard some horror stories about the dutch smart meters.

Apparently the billing system provided for 7 'counters' and hence the smart-meter had to have exactly 7 counters.


In my opinion, the best way to do it is ingesting events in a relational db with an idempotency key to prevent duplicated events. Some companies don't need real time billing data, so you could just use it to aggregate these events with the aggregation function that fits your need for a certain feature. Thus, you have the flexibility of (i) calculating pretty much everything you want, (ii) give granularity of usage to your customers and (iii) be closer to real time (you play the query anytime you need it).


I was also consulting for a utility company. Billing was easy, we used SAP (lol). But CRM (where we also used SAP) was a nightmare. The different processes where a nightmare. I mean utilities are amazingly complex logistically. New metering for a single house build, when it's ordered, installed, etc.


I work in a closely related space: parent's comments are accurate. Some of these items can be solved with better forecasting models, but the issue is that almost no-one actually cares about 'better' forecasting at inidividual meter levels. Sysytem-wide (or at least substation level) forecasts are well studied for supply/demand considerations (at a minimum). Also, the variability in consumption patterns at an individual level are large. IOW, one might be able to generate forecasts accurately for 'typical' residential consumption (at a per-household level), but commercials can be very different. Ideally, IF (and this is a BIG if), forecasts for "most" premises was accurate enough, then one could make the claim that the missing data shouldnt really cause a recalculation of bill once the actual consumption data comes thru, except in cases where the error is large. Guess that means that the error check/correct process needs to continue to exist.

The meter being replaced shouldnt be a major issue, this is relational (at least we treat this all as relational) data that should be captured (by the customer information 'CIS' system), and should be available with a 1-2 day delay. Similar argument applies to other relational aspects of the premise under consideration. Not saying that those are easy (more ways for this process to have gaps).


I've worked at two companies in their payment systems. This post does a good job of mentioning many of the billing system challenges. I think it understates how difficult and important idempotency is to maintain at scale with multiple teams though.

Additionally, it didn't mention financial regulation changes. India had changes sometime in the last few years which required whole new systems built that were specific to India customers. One example of complexity is this: does the system apply to customers who are in India or does it apply to businesses who are in India (the owners might not be)?

He also did not mention fraud, which is basically uncollected spend that results in charge backs typically. If you get too many chargebacks, you'll get fined. If you get even more chargebacks, you'll get kicked off the card network and will no longer be able to accept cards within that network.

Post pay (customer gets the product and pays afterwards) models are subject to more fraud, uncollected charges, etc.

There's also the fun of dealing with downstream payment processors. I haven't worked with a processor or bank yet that was very reliable. Sometimes they'd return 500 http codes and then process a payment, requiring manual intervention. Sometimes that happens with files of payments because most banks deal with files for transactions.


> Additionally, it didn't mention financial regulation changes. India had changes sometime in the last few years which required whole new systems built that were specific to India customers. One example of complexity is this: does the system apply to customers who are in India or does it apply to businesses who are in India (the owners might not be)?

IIRC, I've read this is one of the reasons that systems like SAP are so popular. They might be an ugly monster from a technical perspective and seem overly complex, but they provide thoroughly tested implementations that handle all that stuff.


Hey @tablespoon, I'm one of the cofounders of Lago here.

Products like SAP do seem complex because of (i) the complexity of the billing problem and all the edge cases, (ii) they were not built with an API-first or engineer-first mindset

We're only at the beginning of the journey, but that's one of the reasons why we thought of open-sourcing our billing API, so that the work could be forked and tweaked to address edge cases if needed (no need to wait for SAP to update the product: and they will never prioritize small edge cases). Also, it seemed to be no middle ground between 'Stripe Billing' (it's great for B2C Subscriptions) and billing systems that could address the millions of nuances of pricing between 'subscription' and 'usage-based' (most companies are a mix of both).


In the SAP R/3 family of products, there is also a component Factory Calendar (IMG -> SAP NetWeaver -> General Settings -> Maintain Calendar, transaction /SCAL) - a component that knows about all the various calendar systems incl. public holidays of the world and the dreaded leap years. If I remember correctly, it's customizable for each customer premise with respect to whether each day of the year is salaried or not.

This is a component classified as "basis technology" relied upon by many modules for billing (e.g. should a daily salaray be calculated in a country for a particular person? Should an invoice be considered "overdue" yet, based on the number of business days passed since issue? etc.).


I am the last one to defend SAP, but they do have an API first approach since some time, check this out: https://api.sap.com/products/SAPS4HANACloud/apis/all Also edge cases are taken care of, sometimes before you even small updates, via notes, mostly tax regulation changes. So yeah, for these things SAP is really made for. If you have a SAP account, you can check out, just to see how many changes are coming in: https://launchpad.support.sap.com/#/solutions/notesv2/?q=pol...


SAP and Oracle are always easy targets. Not sure about Oracle but SAP has a relatively newer (10 yr old) subscription and usage billing product - https://www.sap.com/products/subscription-billing.html


It's not like SAP/Oracle/etc have some magical power. They just pay attention to these things, build the product, THEN release the product. And initial releases are buggy just like most software.

So more agile players can get involved more quickly and entrench position.


Hey @Brystefor, I'm @Rafsark's co-founder. We could have spent more time on idempotency, indeed. We have not deep dived into the 3 other aspects you mentioned indeed, we're actually planning to 1/ open-source the articles and add additional challenges as they are mentioned 2/ open our roadmap based on these inputs as well Thanks for sharing! Will ping you back here when it's live PS: we had not thought of financial regulation changes indeed!


Oh, I didn't even realize this was a blog for an open source project. I assumed it was someones personal blog.

Two other things that you may want to consider:

1. PCI compliance. If you're involved in the storage of payment information, you may want to look into these regulations.

2. Regarding point 3 of your mission "...only integrating with specific partners to lock you in (e.g., Stripe Billing only working with Stripe Payments)...". I think there are many companies out there who want to integrate with more than a single payment provider (eg Braintree, Stripe, Adyen, etc) but run into this problem. It increases the cost of payments due to the "rent-seeker" charge method they use. Providing a design framework for how a single company could integrate with multiple providers could be beneficial. This is something I've considered trying to build, but I think it could fit with this initiative.


Hi @Brystephor! I'm Anh-Tho, one of the cofounders of Lago.

We're building an open-source alternative to Stripe Billing indeed, but we want to share our personal experience, hence the personal blog post.

We'll open our repo very soon and our public roadmap too. Thanks for adding the 2 points, we're working on 'a design framework for how a single company could integrate with multiple providers could be beneficial.' indeed!

Will ping you back here when we're live!


Wait until you get into the complexities of taxation.

If you think you know them, pay attention to https://twitter.com/aotearoa_ben/status/1526786701750050817.

That's a law that changes the tax rate if

1. The purchase happens in Texas. 2. The payment cleared in a specific 2 day period. 3. The item cost < $100. The cost of shipping has to be included in the item's cost. 4. The item falls into specific categories that almost certainly don't match whatever categorization system you are attempting to use.

Seriously, calling out just one example, https://comptroller.texas.gov/taxes/publications/98-490/scho... points out that a kit containing school supplies qualifies as long as the cost of the qualifying school supply items in it exceeds the cost of the non-qualifying items. I guarantee that you don't have a category for that in your accounting system...


Our team was building a telco operator in the U.S. We already did that in several European markets and in Australia - so we assumed that we were mildly proficient in taxation issues. And then came the U.S. - a complete nightmare of overlapping rules that sometimes are not properly published. In essence you cannot do this on your own - you definitely need a tax service provider (like Avalara). Funny thing is though that you still are pretty safe to be noncompliant sometimes somewhere and the provider will not take on the liability if their data / computation was wrong. In short: a mind-blowing disaster. And you had to PRINT your tax forms and SNAIL-MAIL them to every jurisdiction that you were doing business in. I mean: hundreds or even more. Every. Single. Month. And don't get me started on the banking system stuck in Stone Age. I definitely would think twice to set up a business in the U.S.


I worked on a tax app. I was amazed to find that some tax tables necessary to compute taxes were not released until months after taxes were due. WTF?


I worked on a tax app. Total nightmare. I remember having to adjust tax rate for a couple parishes in Louisiana and a city in Florida for example. Trying to come up with a readable, testable, way to get that done was something I tell ya.


similar. worked on invoice app having to apply tax, and working with existing foxpro(?) tables that were in postgresql. Trying to reverse engineer without having access to original code, and make things work "like they used to". Someone manually pulled down some free tax thing once per month, but there were so many exceptions that people knew about that were just hardcoded in to the code that I had no access to that I could never make it "just work".

We transitioned to using an external tool (taxjar, IIRC, but might have been something else). Was a pain to feed them data to get back tax data, and.. so many exceptions and exemptions I had to learn about. Weeks later I get "all these numbers are wrong".

I dug in and went through logs.

"no, these are right. taxjar is adding x% on ABC and shipping".

"We don't charge tax on shipping".

"Well, taxjar is doing it, because it's how you're supposed to do it in this state".

"We've never charged tax on shipping"

"Well... you've probably been non-compliant for years then".

I think when they'd started 'computerizing' this back in the early 90s, shipping costs weren't taxed. Or... (IIRC) the amount of shipping cost taxed was proportional to the amount of taxable goods being shipped in the state (vs out of the state) and when they started, what they were shipping wasn't mostly taxable (like... some of the main items were taxable, but the supplies weren't, or something similar). And... if they were shipping to a non-taxable entity (non-profit) then the in-state shipping charge wasn't taxable, but they were supposed to pro-rate it based on the proportional distance in the target state, if that state charged tax on shipping, then track that and remit to that appropriate county/state authority.

These were some of the rules that a variety of people had tried to describe to me both what they thought was supposed to happen and what they thought the original software was doing, but... they could never find 2 examples to line up to confirm or justify those explanations.

Another example: certain items (chemicals) were taxable in some counties and not others, and might be taxed differently based on whether they were being 'used' by technicians vs delivered to a customer (again, different counties had different rules - and some of these rules were indeed confirmed by digging through taxjar and their support and state agency websites).

When this company started, it was one location in two counties in one state, and as they grew, the specifics grew (and also changed yearly) and what they should do became lost to everyone.

Moving things to an external hosted tax-as-a-service system was supposed to 'simplify' things, but it just introduced a lot of turmoil and headache, because routine numbers people were used to for years were suddenly replaced with "new" numbers which no one trusted.


We should demand governments to maintain and provide an API for their tax codes. Let them have skin in the game. If the taxes are incorrect, businesses can point out that it’s the government API’s fault.

Completely developed in the open, and with full test coverage.

This should be required to collect taxes. No API? No taxes. API down? Tax waived.


“We” the engineers (working for the government) still have to develop those APIs and the backends that power them.

The only way to reduce the pain instead of just shifting it around would be to massively simplify the source of truth - tax codes and laws.


True. One is a more realistic reality though. And it results in this thing only being built once instead of a massive headache for all.

I have to imagine government workers are already writing software to identify if we’ve all implemented it correctly, when the could have just given us the code/api up front.


> I have to imagine government workers are already writing software to identify if we’ve all implemented it correctly

Because of the jurisdiction issue, even if each local (and for each location all relevant governmental tax authorities) had a clear API, interacting with them all, and dealing with the interactions between them recursively, is itself a large problem.

And from what I've seen from smaller jurisdictions, the automation software is all about simplifying a human-in-loop workflow rather than full machine-driven rules.


Since we're talking about a hypothetical solution, in theory this would have a spec and potentially even be further centralized.

I could imagine a world where the IRS ran the tax software for every taxing entity in the US. Each jurisdiction would have to configure their specs or potentially even codify the business logic. International would likely be tricky but it's similar to what is being done with ACH and other banking systems that are slowing moving to international. So, it's not impossible there's just not much of a push for it.


> reduce the pain instead of just shifting it around

Nay. Shifting pain from thousands of projects to a single org obviously reduces overall impact.


You might be interested in [Catala](https://catala-lang.org/en/), a domain specific language for implementing legislative texts. Iirc, they have worked on the french tax code, and are starting on the us one now.


Gee, that's bold, I feel like "try writing in plain English" is already asking a lot of the legal profession (fortunately some of them are taking care of asking it of themselves). I wouldn't want to push it!


Here is an irony. If most contracts were rewritten in plain Spanish, they would be understood by a higher portion of the general US public than the current legalese!

If you believe the statistics on literacy and multi-lingualism, it's a tossup as to whether Chinese or legalese is more widely understood.


True, but if the lawmakers writing the tax code bear more of the pain of the complexity of the tax-code (i.e. get complaints and warnings of their own devs that have to implement that API) then they might write better tax codes.


Yes, I think that's the point.


That is the point of the tax code, to be able to use it for political wielding of power e.g. the Chicken tax because LBJ was pissed at Europe that caused a 25% tariff on light trucks imported to the U.S. based on a retaliation for European tariffs on American chicken..


The US tax system is the legislative equivalent of spaghetti code / big ball of mud.


Every tax system is. Because the experts who help to develop it are the same people who get paid to help you file your taxes. Making it easier would lose them income.


Nah it’s not a US thing; tax codes are the same most everywhere.


It is a combinatorial explosion problem. Each taxing authority probably isn't too different from the norm, but in the US you are more likely to encounter many different taxing authorities that overlap.


Some are worse than others mind you, at least for certain consumer facing things that most are discussing here.


I have never seen one quite as byzantine as the USA.


Come to Brazil, we have taxes on products 'circulating', moving your products between logistic centers is OK, but if the product is lost or stolen, taxes are due (as if it was a sale).

Some tax regulations were considered illegal by higher authorities, but that won't stop a tax auditor in a local level charging them and potentially closing down your business.


Why would you do that when you could just lobby the government to make the tax code even more complex, then sell a service to handle that complexity for you under the threat of jail time if you get it wrong?


You're being facetious but I don't think this is a radical idea. Government already publishes the tax code in English language. This is just formalization of the tax code in computer code: More precise, testable and provable.


What makes you think that this is being facetious?

https://www.propublica.org/article/inside-turbotax-20-year-f... has a good overview of how companies that provide software for calculating taxes guarantee a market for their products by keeping the process of calculating taxes more complicated than it needs to be.


In Bangalore, India, workers put on Smith face masks as they posed for selfies with the man himself. Fittingly, the tour culminated in San Diego, the home of TurboTax, the software that transformed the company’s fortunes. There, Smith arrived at his party in a DeLorean, and as he walked a red carpet, cheering employees waved “Brad is Rad” signs. To Smith’s delight, his favorite rock star, Gene Simmons of Kiss, emerged.

I have no words to describe what I just read. That's straight out of the TV show Silicon Valley.


Is it possible to do all of that without being so embarrassed at yourself that you die, or do you have to work up to it?


Damn. I'm with you, this needs to be corrected.


Tax codes are hard. “Get the government to build an API” now you have two problems….

Source used (non-US) government api to do with legislation. The freaking ambiguity and cludges. Luckily they had a team of good people to help.


This actually sounds like a good use for smart contracts if we switch to a CBDC. Basically you pass your transaction through whatever state’s contract and it will automatically do the tax withholding for you.


Good luck with getting popular support for that one!


I used to work at a company that managed the systems for back of house in fast food chains. We moved customers from excel to an online system. Omg when it came to doing labouring in the US. It’s scary.

If a person clocks in at 9:52M. The company only has to pay them from 10am. But in other states they must pay from 9:45am. There’s so much complexity in the us due to all states being different.


It's so much worse than that. Some payroll tax laws vary by SCHOOL DISTRICT of the employee.


Much like airport taxes around the world are also different and can change at any time.

It is an interesting challenge though, to create software that is easy to maintain while capturing all these complexities.


Even when the laws are comprehensible and you're not running at scale, existing accounting systems make supporting simple new requirements difficult.

Imagine a US LLC setting up their chart of accounts to track IRS deductible expenses. Coding expenses into this 17-account schema in the normal course of book keeping means taxes can be filled by copying the account totals into the corresponding IRS form field.

But later, the business wins an federal contract which reimburses certain "R&D" expenses and possibly some "overhead" expenses. Now not only must every expense must be categorized with one of 17 expense categories for the IRS, but it must also be categorized with one of 3 categories for our federal contract purposes. But wait, a small business loan application may require profit and loss broken down by a different set of categories. New and local tax authorities may impose other categories.

One way to address this complexity explosion is with multiple instances of the same accounting software; each imports the same bank account statement, then justifies it against a chart of accounts. But many businesses will break out Excel and hope there aren't too many expensive errors.

If new regulation is inevitable, why does our software pretend otherwise?


Wait until you deal with Brazil. Billings systems are not hard they take engineers that are thoughtful and long-term thinkers which is not conducive to today's software developer hires. Which is why posts like this are made, Its a market ripe for dominance.


Just do what eBay does and just charge sales tax on everything!


* Cries in brazilian *


Implementing and maintaining billing is my daily job for the last 6 years.

I would also mention topics like: - Reporting (various data aggregation and audit reports) - High reliability, nothing hurts company more than unability to bill their customers - Rounding, this can be seen as a subset of "taxes" but it's much more complex - Locating user, also can be seen as a subset of "taxes" but user can have country A set in profile configurations, country B on credit card and country C geolocated from IP address - Timezones, issue everywhere but when we talk accounting it's super important - Talking to moronic payment gateway providers. This is the biggest ones. I would love to just flip Apple and Google like Linus flipped of nVidia. Proprietary, poorly documented, "find out yourself by weeks of experimenting" bullshit with no easy way of getting technical support even when you bring those companies millions per year. Things don't work, deprecate monthly or weekly and expect you to be always ready to make changes. Some implementation make zero sense at all with complete paradigm shift of handling payments like between Google in-app pay and Google subscriptions.

But as my SO says "Don't cry, it pays well"


Yeah, Audit for the win.

And it's not "billing", it's Compliance + Bill Presentment (marketing!) where we are always trying to find the most profitable local maximum of explainability to Sales, Customers, and Management/Enforcement while also having hard checks to prevent or at least mitigate losses.

No one wants a Knight Trading excursion....


Wow, thanks for sharing! We found out 'billing engineers' are very rare and really demanded, as it requires to be very detail oriented, technical, and business processes oriented. But even if there is high demand (and I guess: corresponding high pay, as you mention), very few engineers are up to the challenge!


Yep you are right, indeed. We could have mentioned Reporting/Analytics for revenue, which is always a huge pain for companies!


Once upon a time I was the first product person at a now-decacorn, and my first task was fixing the billing system. It was quite the monster, and we ended up implementing a combination of Zuora and an internal system, as there were some parts of the billing model Zuora couldn't handle.

I came away from this with one big lesson - if you're considering a complex billing model, consider the engineering implications first. With most products, engineering feedback gets taken into account - often product proposes something, engineering breaks it down, product realizes that feature x is vastly more complicated than they thought and not worth the effort, and the requirements are changed to simplify or remove it.

The one thing that never seems to be true of is pricing models - that decision gets made at the very top and handed down with no chance for feedback. I think that if it the folks designing the billing system realized the costs, they might simplify things. If the complexity of your billing system means that 3% of your engineering team (plus additional folks in support and finance) is going to be working on it forever, but if you simplify it a bit you could keep 90% of it and only have 1% of engineering working on it, that might be a good tradeoff - after all, that leaves you more engineers to build features, which should drive additional sales. Unfortunately, that analysis never seems to get done up front and the cost is only understood after the billing system is deeply integrated into everything and would take an unpalatable amount of effort to change.


Yes, this is a big thing - there needs to be a clear model of how variable pricing and discounts are going to work in this company, with the sales team being able to apply the actual amounts of any prices and discounts, but not arbitrarily changing the model.

It doesn't work when the model is too simple or restrictive, which will simply result in the model being violated; you do need at least the ability to customize pricing for individual customers, and for specific invoices, but other than that the decisions (e.g. whether you apply invoice discounts as percentages or specific amount or both) can vary but you just need to pick any reasonable option and stick with it.


Yeah, this is tough because when the VP of Sales knows that one particular change will close the deal, they're going to push for it no matter what the cost. In reality it may be better from a big picture perspective to give a concession that is actually better for the customer and worse for the company than x as it relates to that particular deal, because the larger concession fits within the current billing system with no custom work, so the overall cost to the business is less.


Couldn't agree more on this. Even if finance, sales, marketing or other deps are involved, billing is an engineering thing! Back in 2016, while building the billing system for qonto.com (european fintech unicorn), we were surrounded by people willing to add complexity to a thing they don't understand. A team of 2 engineers building it ended up in a whole squad called "Pricing Cross Functional Team"... even the name of this team was complex :)


Just wait until you meet billing's angry roommate: invoicing.

In the US, an invoice is just a weird PDF that you might glance at before sending off to your accounts payable team. But in other countries, especially those that use VAT style taxing systems, an invoice can be a legal document that expresses costs and taxes in a legally prescribed way for accounting purposes. Many countries have prescriptive rules over how you invoice, who can write invoices, when you can invoice, what goes on an invoice, etc. And every country does it differently.

Are you building a service that charges incrementally many times per period? Or even worse, refunds incrementally? You might be sending a truckload of expensive accounting papers to your customer every month, one per transaction. And each of those pages was printed using government prescribed software from oddball vendors you must use or else.


Many years ago I led a billing team at Yahoo! responsible for billing for premium services in European markets.

My team existed for the sole reason that the European business did not trust that the US payment services team understood European needs, and the "invoice is a legal document" thing was one of them. I spent so many meetings repeating to the US team that no, we could not switch to their invoicing as long as a new software release might retroactively change the template used to show already issued invoices (e.g. the registered office might change, or the VAT number).

We didn't have to deal with printed invoices thankfully, but we did ensure we produced each invoice once and stored it, so that the European finance team could sleep at night.

At the time Yahoo! had 8 different payment platforms. Some of those were due to weirdness around acquisitions and Japan (which was a joint venture), but apart from mine I believe two others also existed because of local weirdness that the local businesses decided it was safest not to let Yahoo! US mess with.

At the time I left, after 3 years of that, we'd managed to get agreements to migrate a few bits and pieces to the US after they'd shown the understood requirements for specific features, but it was like pulling teeth.


I'm Anh-Tho, one of a co-founders of Lago, thanks for your comment! I was the country manager of France for a US Sequoia-backed company before, and I never ever managed to convince HQ to work on EU invoicing, I think they just did not want to open the pandora box you just described!


I definitively am interested in a couple of months. I rather not build our own if we don't have too.

Do you have any early guides or api docs to take a look at. Our use case.

1. Base service fixed packages (based on users with volume discounts)

Extra costs 2. Extra storage space usage 3. Add on services (fixed or usage billing) 4. Invoicing third parties (reselling content)


And even worse - your biggest customers won't even smell your invoice unless you enter it line by line into some ancient SAP system developed 20 years ago, where everything cloud related is classified as telephony except storage space which must be classified as filing cabinets (movable) or your invoice will be (very slowly) rejected.

And if it is rejected you have to enter it all again by hand; editing isn't a feature.


I said, over and over again, that moving away from goats and salt as currency was a mistake, but nobody listened to me.


But the different conversion rates for each kingdom/duchy/province/parish were a nightmare even then.


I don't get what you're talking about. Our biggest customer reluctantly agreed to accept preliminary invoices for acceptance over fax instead of signed-for mail, provided that we also ship a paper version with a red stamp promptly.


Wow, haven’t lived this pain fortunately… but totally understand how hard it is to try building a « modern pricing stack » but being forced to create invoices in old softwares like SAP…


Yeah - and simply creating an invoice is creating taxable income in EU - if customer drags their feet you still owe VAT to the taxman.

If you want to change invoice you have to make a special "correction invoice" because changing "real invoice" is a criminal offense - fun things :)


This is a property of how VAT works. In the common scenario, the seller is effectively "tax administrator" for the buyer (much like employer pays payroll taxes on behalf of employee), therefore an invoice triggers two independent payables: 1. Seller to taxman 2. Buyer to seller

Neither seller cares how you handle your payments to the taxman, nor taxman cares about customer payments.

The second part regarding credit/debit invoices is again a property of "append-only double-entry bookkeeping". You fix wrong credit/debit entry with reversed debit/credit entry.


To spell that out: if you made a mistake in an invoice or negotiated a new total or an extra item (or had to drop an item) after creating the invoice, you now need to first create a correction invoice cancelling out the invoice you created and then create a new invoice. All three documents must have different numbers. Oh, and invoice numbers must be sequential and continuous.


Many years ago, my dad wrote an accounting system for the company he worked at to automate some of this. He spent ages trying to convince the tax authorities that the database was sufficient, and that there was no point in printing out an extra copy of every invoice and gluing it into a book. The idea is it'd make it harder for you to retroactively manipulate invoices.

They eventually saw the light, but it was a long slog.


In the recent years Germany has started allowing companies to no longer issue print invoices. But you still need to keep a copy of every generated invoice.

Initially "digital invoice" meant that you had to get the invoice cryptographically signed by the same government agency also in charge of literally printing money (or an officially licensed company) and then ideally send it using the now mostly defunct monstrosity that is De-Mail because it made guarantees about end-to-end encrpytion and sender/recipient authentication. Luckily this is now largely irrelevant and most companies just send regular PDFs via e-mail and/or make them available for download.


Yup, it's a nightmare. Invoices needing to be in specific number sequences. Any corrections need to be dealt with by reissuing a special "corrective invoice". Don't even get me started on geocoding for taxes. Urgh. Also, when you've got the invoicing right, you've got to work out how to do the feeds to the accounting systems so that it all posts to the right place.


Yep, fortunately in Europe we can issue credit notes, making our job easier. I do think giving the possibility to send charges information to a webhook is a great way to start: this data can be plugged to any invoicing system (or any tax, payment providers) who are trying to solve those things. The hardest part for me, when working in that fintech, was to build the whole logic to trigger the invoice, not so much the invoice itself. But this company was a EU company, probably easier to work with invoices


I’ll add that the accounting side can get complicated really quickly. You end up with lots of different scenarios. Earned but unbilled: out of bundle call charges, for example. Billed but unearned: a paid in advance monthly fee, for example - this may need to be recognised as a percentage per day. Etc etc. So much to take into account when feeding to ledgers.


A credit note. How the ** do you handle it in the US without issuing credit notes in your bookkeeping?


We issue a credit note*

* Except (these aren't the right thing to do, but I've seen it in the wild many times):

- When someone in sales issues an invoice with a negative amount on line 3. Hey, if I take payments with invoices, why not give back?

- When someone agrees to settle up by providing free services and throws it on the next bill.


> When someone in sales issues an invoice with a negative amount

I worked briefly with a tax-saas, and to handle refunds I just... did a negative amount. It worked! Except.... I hadn't looked closely. Their API removed the negative. A charge of -$7.92 was just a charge of $7.92. Their docs indicated "we don't accept negative numbers - use a refund transaction instead" but... the API accepted the negative number, then changed it to a positive, and kept running. No error code returned. Really really poor experience, and I hope they've fixed that.


Invoicing gets even better in countries like Portugal, where you have to send to the tax authorities every invoice that you generate.


In Italy too. They specifically developed an XML format for that, and each time you issue an invoice you have to (well, with some exceptions, but they are gradually removing all of them) send a copy to the revenue agency, which will forward it to the recipient (and keep a copy).

While it is a bit inconvenient, though, I don't think it is a bad idea. I am pretty sure it helps a lot making the life harder for people evading taxes, and to some extent the availability of a standard machine-readable format makes it easy to do accounting. My wife sells videocourses online and I developed a thing that automatically issues invoices as soon as people buy a product (with some human supervision, mainly to avoid the machine doing havoc is some exceptional situation happens; normally it is just "click a button to send the invoice"). The invoices are then automatically available to the accountant for doing accountant things.

You can find the XSD here: https://www.fatturapa.gov.it/en/norme-e-regole/documentazion...


They should rather give a service to issue /cancel an invoice and people will happily use the service


Not sure it is a good idea. It might be more flexible when you issue an invoice and have to modify it, but first it would enable fraud, and second it would mean that you have to monitor incoming invoices to see if anything changed and account for that. I think it's better to ask people to just emit correct invoices: if you need to invoice a bit more you issue another one, if you need to invoice less you issue a credit note. If you can't do that you need to work on your business processes.


And then just wait until you meet Invoicing's annoying cousin Purchase Orders (PO). THere is a PO. Can I pay this in multiple invoices ? Sorry the PO is not approved yet. We cannot accept an invoice. Sorry the Invoice doesn't have reference to our PO. Can you fix that please ?


Annoying but necessary if you dig into it. If don’t we’ll you avoid paying random invoices emailed to your invoice processor!


Hey @codegeek! I'm Anh-Tho, one of the co-founders of Lago.

We're working on making the 'CPQ' : Configure Price Quote journey simpler, and this includes the PO dead end. We're on a mission to make the pricing stack more open and customizable for companies (hence the open-source angle), and we're starting with the billing system. Join the journey, we'll share our repo soon, and hopefully we can address these painpoints faster with the community contributions too!


Yep. Invoice software is country specific. One can sell invoicing software to several countries, it just can not be the same software. And no, you can't just abstract the differences, because the invoices focus changes widely.

At least we are in a situation where the seller is almost only subject to the laws of the seller's country. There are some exceptions, but one can mostly deal with those (the EU has plenty, the good news is that if the seller is not also on the EU, most do not apply).

Also, I haven't seen a country accounting system that made accounting papers inherently expensive. Some make refunds very expensive, but not the papers. It's more a matter of badly designed software and processes.


I work on a payment system that invoices consumers on behalf of other companies and pays out the money to these companies by splitting the payout on multiple partners (luckily I do not handle the support and follow-up). The system have ability to refund invoices and customers some times end up paying more than or just parts of the invoice. On top og all this it integrates with several old 90s systems with poor datetime handling and poor uptime (some times a windows xp box in the cudtomers offices). It also handles card payments and over all, by far the hardest thing to get right is the invoicing. Its just so extremely fuzzy and time sensitive.


And most big companies who agree to pay you net X days, that usually is after the presentation of a correct invoice. The invoice doesn't get to them, or it isn't right the clock doesn't start for them to pay you. Getting invoices wrong can quickly result in major cashflow drops.


I'm Anh-Tho, one of Lago's co-founders here. Indeed, from our experience people see 'billing or invoicing' as a monolith block, whereas there's a whole 'pricing stack', billing > payment > invoicing > cash collection + accounting + CPQ + sales commission. For cash collection, we mentioned the tool used by Lattice called Upflow which helps to manage account receivables.


Hi! We decided to aggregate all costs of a "billable period".

Imagine you bill your customers monthly (the billable period), all the charges (usage-based features + subscription) will appear as line items of a single invoice.

This enables you to gather all the fees of a period into a total invoice, but still be able to provide granularity to your customers (breakdown of all the fees to be paid).


Hi, customer here. We'd like to change our billing period to start on the 31st of every month, and we'd like to pay 90 days in arrears, and we'd also like you to invoice us ahead of the billing month. Oh, and we're big enough that losing our business will end your startup, so don't get it wrong. Thanks!


I'm Anh-Tho, one of Lago's co-founders here. Thanks for sharing! What did you do then? I'm curious!


Hey, Anh-Tho, just as a point of feedback, since you're leaving a lot of comments in this thread, it's not necessary to introduce yourself in each one. Most folks will be scanning the whole thread and seeing your introduction gets repetitive and breaks the conversational tone and makes it overly commercial. Thanks.


I fear if they did not they'd be accused of something else. I agree with your observation but I don't think there is an obvious solution as you imply.


They can disclose their interests without the repetitive intro. But for the above comment, just asking a question, even this is unnecessary.


I agree it was repetitive, but indeed, I did not want people to think I was trying to pretend to be unrelated to the post.

Thanks for the constructive feedback!


Wait, who are you again?


I am Anh-Tho blabla... just kidding :)


you're welcome


Noted thanks! It's my 1st post on HN, so I appreciate the feedback.


I was illustrating the point that customers are often unreasonable about billing, and there's no magical solution that fixes the problems. The answer is really to try to make sure you're not overly reliant on a single customer. That's hard as a startup though especially if you're in B2B.


Just one of many reasons its easier to do business in the US


Hey cm2012, I'm one of the cofounders of Lago (my business partner wrote this post). Actually even if you're incorporated in the US, if you serve a client in EU for instance, the VAT rules of EU apply to how you're supposed to invoice your customer, beyond a certain transaction limit. So... unless you're based in the US, and only serving US-based users, it gets complex very fast!


That's their point, they said "do business in the US". You're saying the same thing.


If you wire your dollars onto US soil, for a service performed in the US by US business, good luck getting the US legal system to enforce upon that US business whatever euro-cucked invoicing scheme is called for.


This doesn't matter. For B2B, if you don't show your EU customers a valid invoice, they simply cannot pay you.

It's not EU governments or post-purchase issues you have to deal, most of the time - it's trying to persuade a large accounting department to pay you unaccountable money. If you can't follow their invoicing requirements, it's just not going to happen.


>they simply cannot pay you.

Do people ever stop and wonder WHY in nations like Portugal the informal economy is roughly DOUBLE of say the US? When every invoice must be available to the government, what actually ends up happening is non-conforming invoices either get made up by the customer or the money magically flows out under some other auspice.

Making it literally illegal to accept an invoice that is legal in the country in which you obtained it borders on logic even my toddler can understand is absolutely begging for bad business climate or tax evasion.


Feature, not a bug. If there's pervasive lawbreaking, then the officials can collect more bribes.


I'm Anh-Tho, one of a co-founders of Lago. I'm French, and I confirm, in B2B, no invoice (that is compliant, not a mere receipt) -> no payment.


Thank you Anh-Tho for your response.

What is the mechanism for French government to subpoena a non-EU business to find this invoice? If the customer (instead of vendor) produced an invoice that matched their bank statements and non-conforming receipt, hypothetically, would anyone really know the difference?


Probably they wouldn't as full audits aren't that common, but audits do happen and most accountants won't care about your purchase as much to personally commit a felony and forge documents for no good reason (accountants do have personal responsibility, and every accounting course reminds wannabe accountants that "the boss ordered it" is not an excuse) so they simply say that they can't/won't do it and if the vendor can't send a satisfactory invoice then they won't do business with that vendor.


How would a French audit uncover anything wrong with a conforming format invoice under the name of some random American company that exactly matched the bank statement, physical goods, and customs paperwork?


Hey notch656a! I'm not a government expert, but the challenges would start with:

- I am Frenchy (a French co) - Buying software from Ricky (a US co)

If I don't have a proper invoice from Ricky to give to my accountant at the end of the year, and I get a tax control, then, I'll be fined.

If the tax ministry notices large amounts of similar fines, they might look at Ricky.

So the risk is more on Frenchy (risk to be fined). But I'd say that Ricky is safe for a while, unless the invoiced amount are really huge.


European bureaucracy is out of control, and that's just the west. In the east and former Soviet states, it beggars belief.


[flagged]


> it's against your freedumbs

You don't even know what freedom is or how fundamental an issue it is to the United States.


I don't think US has many relevant freedoms left. Some people cling very hard to legacy 18th century ones while the Moloch just gobbles up all the new ones stemming from progress.


We do in theory at least. But in practice... you're right.


>Do people ever stop and wonder WHY the US is the biggest tax haven on earth?

I consider this an advantage of the US, not disadvantage.


That american exceptionalism approach will work with private citizen customers. With business customers, leaning back in the legal framework encouraging tax fraud will not fly. They just won't buy from you then.

That's why any US company wanting to do business in Europe either has an European subsidiary, or follows something that's compatible with EU invoicing schemes.


>. With business customers, leaning back in the legal framework encouraging tax fraud will not fly.

Yes which is why in nations like Portugal, where any business invoice may be subject to accountability to government, there totally isn't a massive informal (read: tax-evading) economy to deal with the fact that the government makes it literally impossible (at least in above words) to accept an invoice legal in the jurisdiction in which it was issued.


People are mixing up things here

It's not illegal to pay an US company with an US invoice in the EU. (HN likes to invent problems where there aren't, and companies do that all the time). It is not even limited to the US, most other countries can't/won't follow the format.

However there might be extra steps in adding those invoices to their accounting. For example, when import, the buyer will be charged the import tax (instead of having it added to the invoice)

The informal economy of Portugal has nothing to do with the above btw since that has nothing to do with American invoices (and you can bet German rules are not much simpler) - not saying the bureaucracy isn't, most of the time, stupid. The US also has it fair share of stupid crap that people have to deal as well but it is "transparent" to most Americans.


You keep bringing this up and it's simply not true. The EU has a single set of VAT invoice requirements that applies across the EU (the MOSS invoicing standard). Any VAT invoice that satisfies these rules is legal in all EU countries (https://eur-lex.europa.eu/legal-content/EN/ALL/?uri=CELEX:32..., article 226).

My company does plenty of business with EU businesses and we use a single standard EU VAT invoice without issue.


It's not literally impossible - the simple solution to standard invoice being unacceptable is to issue a different invoice that matches the requirements, and almost every vendor is happy to do so in order to make the sale.

Also, the informal tax-evading economy relies on personal contacts and 'mutual understanding' - it's not easily accessible to a foreign vendor, and when foreign vendors do want to access that economy, it requires much more adoption of local customs than just making a different format of invoice.


Yes literally impossible to take something in a format legal in the US, that doesn't match the foreign requirements, based on European testimony above (the exact truth of which IDK).

Informal tax-evading (when the seller isn't evading, just the buyer) doesn't require personal contacts. This is just naïve thoughtless statement. It happens all the time. Example: business owner goes to Panama where they know no one. Buys a pallet of llama wool, seller gives a non-conforming receipt. Buyer goes back to France, imports the pallet as "cotton" and creates a fake invoice showing the Panamanian sold cotton. Seller then sells llama wool on the streets for cash, marking in the books that they sold "cotton" for significantly less. They then use the on-the-books "cotton" proceeds to buy a shit-wagon car or something, and fix it using the unrecorded cash on the side. They sell the now nice shit-wagon and record the proceeds as profit. Now all the money for the llama wool is accounted for.

Cross country tax-evading is only assured in this case to require personal contacts when the tax evading happens on _both_ sides. When it only happens on the EU side, there's no need for personal contacts on the US side.


> Buys a pallet of llama wool, seller gives a non-conforming receipt. Buyer goes back to France, imports the pallet as "cotton"

Customs catches him (distinguishing cotton from non-cotton is surprisingly easy, so is forged documents for invoicing and freight papers, especially given that Panama is an unusual country to export raw cotton - or raw wool - from), he is now in prison for 5 years for tax fraud and document forging, llama wool is confiscated and auctioned off to legitimate sellers.


Maybe you can come up with a better example then :) Here in the US pallets of cocaine regularly make it through customs. Our customs officers are generally one rung above high-school dropout. They'd generally be lucky to distinguish heroin from a pallet of sand, let alone cotton from wool.

Not in the export/import business myself, just passed through customs enough to see just how truly stupid and vapid these officers are, combined with the truly insane amount of import/export volume each one is responsible for. In the US distinguishing types of fabrics and ensuring various legal substances that look vaguely similar actually are what they say they are is near the bottom of the priorities list. I don't have a lot of experience with EU customs (except see last paragraph).

I know people want to believe customs is some highly intelligent caring group of individuals who are sleeplessly guarding the country and the purse in a carefully planned and executed manner with a fine-toothed comb. Anyone with this belief has probably never crossed an international border. In practice it's more like the mental equivalent of a 4 year old set loose with a completely unreliable "drug dog" that alerts on a grandma who is ruthlessly interrogated while one guy with llama wool and 10 pablo escabars pass on the other side.

I can recall one occasion where I stepped directly from fucking IRAQ into EU and not a single customs officer even looked at me. Not one. Just a stamp without a single question. I could have been carrying literally anything and no one cared. I could have been coming from Iraq for any reason and no one gave a fuck what that reason might be.


There isn’t a massive tax evasion economy in the USA?


Yes. It would be even larger if US companies couldn't accept foreign invoices without being in a special format. Does tax evasion in the US somehow disprove tax evasion in Europe?


The difference is marginal at best. Most of the requirements are obvious and never/rarely change, like your address or the date. And adding a an incrementing number to a set of documents isn't exactly rocket science, either.


Protecting consumers is always bad for business


In this particular case, do you think non-US governments could protect the interests of consumers in their jurisdictions in a way that is less unfriendly to the businesses that want to sell to them?


Not even legal issues.

In my experience invoicing, and really all 'printable' documents is the land of "oh someone did something wrong put X on the document". And then again and again and again... And "this is two pages long that's too much" and on and on ...

It's bike shedding insanity with no ideal or end in sight. Every change is arbitrary with no real measure of success.

I swear the only time anyone LOOKs at these documents is to bitch about them.

Oh man and just as if I summoned it someone just sent me a ticket about something on an invoice.


Just because you don't understand invoicing doesn't mean that the concerns that appear trivial to you are actually trivial.

From the perspective of someone who works with the people who process the invoices, putting "X on the document" and "two pages long" can be a huge deal when you have to deal with hundreds of invoices or when "X" can mean that an invoice must proceed down a different processing path. These aren't bike shedding concerns; they affect the actual processing of the invoice. In many cases, these concerns affect the validity of the invoice, meaning that the recipient's obligation to pay (or the associated deadline) is not triggered.

And doing something wrong on an invoice is never bikeshedding. An invoice is a legal contract, and in the E.U., an invoice is an unmodifiable document.


They're trivial... when you make the same change to add something, remove it, add it, remove it and hear that it's all being done "because X" and "X" never stops, it's clearly not working and the changes are trivial.


No, the changes aren't trivial.

It sounds like you simply don't understand why they're doing it and you just assume that they're trivial because you have no knowledge of the domain.

Off the top of my head, I can thing of a dozen reasons for why they'd require something to be added, then removed, then added, etc., from one invoice to the next.


> dozens of reasons

Then why not list them right here, right now?


- Changes prompted by common vendor mistakes: DBA vs Legal vs Tax name, Tax vs business ID vs tax account id, projects vs Departments vs Accounts vs Budgets

- Changes prompted by compliance needs: business type (which governs the type and level of compliance required in most jurisdictions), changes to invoicing laws, changes to tax compliance laws

- Changes prompted by business needs: changes to the company's PO/invoicing/expensing process, changes to the company's financial, ERM, or payment processing software or service provider, one-off large expenses that aren't adequately handled by the existing data form; changes to financial modeling

- Changes prompted by marketing: new logo, design, or template

- Changes prompted by legal: new court rulings, etc., affecting legal's approval of the existing template(s)

- Changes prompted by employees: form is too complicated, form is not complicated enough, form doesn't capture my department or project's needs

The problem with an all inclusive form is that it is too complicated for most vendors and they end up filling it in wrong (requiring several rounds of review by the payor company), but too simple a form and you end up needing to make a lot of changes like the one the parent was talking about.

You could do multiple templates, but that just multiplies the work, and makes it more difficult for employees to figure out which template they should be using, and so they'll all end up using whatever random template they find first leading back to the One True Template and you're back to square one again.

And yes, all of the examples above are based on real life examples.


> It sounds like you simply don't understand why they're doing it

I don’t know why you would think that considering I work with them and you do not…


I work with the A/R and A/P teams all the time. It's a large part of doing tax in-house.


>And every country does it differently

Seems like the real world is just choc-full of Liskov Substitution Principle violations :P


> You might be sending a truckload of expensive accounting papers to your customer every month, one per transaction. And each of those pages was printed using government prescribed software from oddball vendors you must use or else.

I guess that's good for employment numbers? There's really two ways to create jobs, innovate or regulate. The US and the EU have apparently chosen very different paths. With every country having it's own special snowflakes laws, there's a reason EU founders always try to come to the US first.


I work at a b2b company with a 25+ years history, and OH MY GOD...

* Sales is payed (partially) off commissions, so they do their very best to sell whatever the customer wants, not what the engineers know how to bill, or that product management has prepared. It's really hard to push back against a contract with revenue 10x your yearly salary "just" because the new features to bill it don't fit together with existing features.

* It's very hard to keep track of which customer pays for what, exactly, and is entitled to which service. Easy for standard products, hard for bespoke products, nearly impossible for shared services (it took us >9 months to sunset a shared HTTP proxy that was in use by just 4 customers that didn't even pay for it...)

* Legacy contracts: imagine 25 years iterating on product design, and keeping the old contracts running (except when the customer leaves on their own). Some of the old contracts are hard or impossible to find, nobody wants to spend their time rummaging through them to find out what exactly a customer is entitled to. A year ago we still had a customer that still pays for 8€ per GB HTTP traffic (web hosting), because that was the going rate back in the days

* Usage-based billing: half of our customers want to be flexible and only pay for what they use, and the other half uses SAP and for them, invoice amounts changing from month to month are the pure horror. So clever sales people invent hybrid models ("pay for what you use, but fixed price until a certain threshold, we'll notify you when you get close to reaching the threshold"). Another source of complexity.

* Usage-based billing: it's basically a topic for the whole company, most departments provide services that needs metering, but metering is usually an afterthought to the architecture, hard to verify for somebody on the outside

* Usage-based billing: for big customers you cannot put all the details on the invoice (our mailing service provider is limited to 99 pages per invoice...), so you need separate reporting to drill into the details. Fire and brimstone will rain down if reporting and invoice disagree on some numbers...

This just scratches the surface. Customers that want to split their bills along cost centers that don't align with access control tenants, different electricity prices based on location, the list is endless.


The article just scratches the surface too. I could have written a book to describe it!

Reporting for revenue is a hard game, indeed. Spent a lot of time trying to reconcile revenue streams in the same company we ended up building the whole billing system. It’s also not easy because metering is not something that is predictable nor easy to make it fit into a finance report.


Basically everything about enterprise is a nightmare for engineers. What amazes me (after having worked at some of the most profitable companies in the world) is just how little intelligence the leadership has about their own revenue or costs, beyond "wow huge amounts of money is coming in or going out". And how many critical processes are implemented manually, by individuals, with personal spreadsheets, on their laptops.


Yup, I work at a tech company and our entire finance & billing team is using excel exclusively (I think our payroll/leave system is the only thing not on excel), where every other team uses one system or another. All of the higher ups makes decisions based on docs created out of all those excel artifacts. I have to admit, their productivity is quite good and they are kings at aggregating and graphing data in sexy ways. It would be a challenge to build them something that actually works better and can adapt quickly to changes (the software lifecycle too slow for them and now they have to deal with some developer types to get something done, so skip all that noise and stick to excel).


It's a difficult topic to tackle; The financial leader has different needs than the tech one. One needs a perfect report with exact match day over day, month over month, year over year. The second one needs to do the math to calculate instant consumption of the product. If we add marketing leaders of a company to it, it brings creativity to the extreme (like inventing new prices, just to use it "as a hack"). Same goal (revenue), but definitely not the same path to success


> intelligence the leadership has about their own revenue or costs

I see the positionning of the CFO right next or below the CEO as a mechanism to let the CEO care about “big pitcure” money flow, while having someone in contact with reality guide decisions and veto stuff that won’t fly in cost/revenue terms.


I have a SaaS idea I’ve been sleeping in solely because building the billing, authentication, & marketing are each an additional 96% of the work that is not building the actual functionality of what I’d want to sell.


Yep, I'm about ready to start implementing one of those very spreadsheets just to start trying to categorize Azure costs.


The reason billing systems are hard is because they tend to just accumulate features/behaviours/requirements. Forever. Although billing is not my specialty I have had to involve myself in it as product owner and business unit manager. Try and imagine the amount of logic, complexity, rules, and exceptions involved in a system that has accumulated perhaps 30-40 years of customer sign-ups. Umpteen contracts, umpteen iterations of each contract, and perhaps 50 or 100 products, each sold in 10 or 100 countries. I now know more about the tax system in South America than I ever wanted to.


I see building or implementing these kind of systems as "discovering" use cases for given scenarios/customers/markets. That's why "billing" is almost too generic - instead "billing for telco in South American markets" - is more useful because those verticals have specific sets of use-cases and challenges.


Yep, totally right. Same at the previous company we worked for and built the billing logic. You start doing it in house because the first version is simple. It’s getting complicated when you keep adding paying features to a product. Often see product managers becoming experts in taxes and accounting :)


A classic, from http://www.canonical.org/~kragen/tao-of-programming.html

> There was once a programmer who was attached to the court of the warlord of Wu. The warlord asked the programmer: ``Which is easier to design: an accounting package or an operating system?''

> ``An operating system,'' replied the programmer.

> The warlord uttered an exclamation of disbelief. ``Surely an accounting package is trivial next to the complexity of an operating system,'' he said.

> ``Not so,'' said the programmer, ``when designing an accounting package, the programmer operates as a mediator between people having different ideas: how it must operate, how its reports must appear, and how it must conform to the tax laws. By contrast, an operating system is not limited by outside appearances. When designing an operating system, the programmer seeks the simplest harmony between machine and ideas. This is why an operating system is easier to design.''

> The warlord of Wu nodded and smiled. ``That is all good and well, but which is easier to debug?''

> The programmer made no reply.


Heh, I'd say the operating system is easier to debug, because the accounting package could literally have "bugs" that are caused by errors in the tax laws, business processes, etc, which cannot be fixed, only worked around.


Which came first? I am pretty sure it was computer accounting systems.

Which is older, the oldest continuously supported operating system, or the oldest continuously supported (computer) accounting system?


The oldest computer accounting system was likely LEO - https://en.wikipedia.org/wiki/LEO_(computer), online in 1951 and doing accounts probably by 1955, if not earlier.

I believe it predate operating systems by a few years. (Wikipedia claims the first OS was the GM-NAA I/O, produced in 1956.)


Applications programming has a complexity of it's own that's distinct from the problems in systems programming.


The linked page does not show it, but The Tao of Programming is actually a book which you can buy:

https://en.wikipedia.org/wiki/The_Tao_of_Programming

I own a copy; it’s great.


OMG love the story, thanks for sharing!


Thoughtful, and made me laugh. Thanks!


Pro-rated billing is one of the many places outside of database query planners where Dynamic Programming is useful! Maybe there's a better way to do it, but well, if you give a man a hammer.. I wrote up how we did this at Equipmentshare back in 2016: https://tech.davis-hansson.com/p/price-wars.html


Billing is a nightmare, and if I had one piece of advice for people building a pay-for-what-you-use system like most SaaS, it's this:

DO NOT bill against your business logic entities. They change, and doing a COUNT at the end of the month won't catch all things which changed or which cost you money during the month. Instead, figure out what you bill for and and when you do that thing, record a row in a DB or into a steam of that "billable event".

Reconciling billable events is much easier to do, and it's tolerant of all the weird edge cases (such as a support person hard deleting data when they should soft delete, which would otherwise throw off your end of month counts).

There's a reason AWS (in general) can produce a log of everything you did which cost you money. It's painful, but it's less painful than the alternative.


Sound advice. Anything to do with accounting, you’ll probably want to treat as an append-only log of events. (Note, you can also use event-sourcing and have your domain entities _be_ events, in a billing bounded context that might make sense. Not usually the first approach I’d recommend though.)

On a similar note, make sure you think about bitemporality as well. In other words, “effective_at” vs. “”created_at”. You might learn that due to a bug, you didn’t record a billable event, and you need to insert it into the history (say, to put it in the correct billing period). But setting the “created_at” in the past brings a bunch of issues; it’s confusing and you’ll soon hit issues where lying about created-time trips you up. (Migrations, reasoning about what version of the code applied what events, etc).

Fowler has good further reading here: https://martinfowler.com/articles/bitemporal-history.html.


If you squint, this is similar to how e-commerce billing is done. You have stock keeping units that represent the logical item being sold, and every transaction clones that row as part of the order record.

This way, if the price changes, the item description changes, the images on the item change, etc, you can still have a record of what was actually sold and how much to refund a return etc.


I think event sourcing / logging is the way to solve this kind of thing.

You have an obvious start/stop checkpoint in the log, processing can be done in batch after hours, no information ever gets destroyed, etc.

Mutable database rows that are used for accounting purposes should sound dangerous to anyone trusted with these systems.


And depending on how you do it, it's not as hard as it sounds - for example, the first minute of "cloud server V1" can get a row in the db, and each "period" that row gets updated with the current minutes.

Or you can log them into tables that get processed into other invoice tables.

This also lets you keep grandfathered customers around as long as you want.


Couldn't agree more! I do think the best way to do it is to log ingested events in a db, and decide which way you want to aggregate a billable feature at a defined billable period. This way lets you (i) keep track of everything, (ii) invoice complex behaviors and (iii) provide great granularity to your customers inside a final invoice.


I've had to deal with subscription based billing, and in my experience it wasn't a nightmare. There are some corner cases, but, those exist in every domain. And the stakes are definitely raised since you're dealing with people's money.

Usage based billing sounds like a nightmare though.

Taxes in the USA are a nightmare too. I have a friend that lives on a street, and for one side the tax rate is 7.5%, and the other side its 9.25%. Literally no company we tested got this right. Even Amazon.


For simple subscription-based billing, it's somehow not this hard, you are right. It also depends on how much plans you provide to your customers, and you still need to scratch your head for upgrades and downgrades.

I truly believe in usage-based billing, so I do think pricing are getting more and more complex over time.

Taxes are a nightmare for everyone also (in Europe too...)


Dont forget price changes. Gotta deal with that. Then discounts, trials, weird durations, and a bunch of other shit I forget.

My advice for anybody thinking of building their own subscription management system is to stop and go look at something like Zuora. You do NOT want to get stuck maintaining some home grown subscription management system--it will never be better than Zuora and you'll always be playing catchup, leaving your company unable to quickly react to the market. All the time spent adding functionality you can buy from a third party is money that you could have invested in your actual product. And worse, the fact you couldn't quickly react to the market because some key feature was missing in your homebrew thing is literally leaving money on the table.


Transitioning to any non-homegrown system is not trivial either, as you'll need some process to reconcile info from current system and some external system. However powerful some external systems are, planning a migration to something non-homegrown is itself a monstrous process. Benefits of staying 'in house' can simply be you're not wasting time trying to fit your existing processes and data in to something external.

That's really great that Ford uses Zuora. But for someone trying to work with an inhouse system that doesn't map to what a system like Zuora can handle... you now have technical work and a lot of political and non-technical work, and a lot of internal coordination to get all relevant people on board to commit to a change.


How often to neighbors on that street ship their deliveries to one another to cheat on taxes?


No company got it right - they all charged the same percentage regardless of which side of the street you lived on - so you didn't have a choice in the matter.


I implemented billing at WakaTime with Stripe, Braintree, and Coinbase as swappable payment providers. It's definitely a source of edge case bugs, but overall it works pretty well. It supports prorating subscriptions across providers, monthly & yearly billing, trails, discounts.

* We don't remit taxes for customers because that would be impossible to build in-house, but everything else was achievable.

* We have tons of logging, and generate transient Quotes on our end, then check that the expected charge matches what the payment provider actually charged.

* Each Quote is reproducible, so if any bugs come up we can give the "create quote" code the same input and see what went wrong.

* What helped when building was hourly tasks running in the background to make sure customers who should be billed were being charged, and make sure we didn't double charge any customers, and then alerts so we could find and fix those bugs.

I would do it again to support both PayPal and Stripe, but it's definitely a source of complexity for the code base.


Hi! I co-led the billing stack re-write at Uber circa 2017 and it was an absolute nightmare. Some of the unexpected hard things that are not mentioned in this article: - Manage promotion was very complex - Modeling money (we used to model it as cents ie 1/100 of the common denomination ... turns out it broke a lot of countries like Japan where the Japanese Yen doesn't have cents) - Timezone is a hellhole

One interesting open source project was https://killbill.io/

PS: Since I built a billing and invoicing startup .. check it out: zenbill.com


Hey thanks for showing me this. What lies beyond the starter plan in concrete terms?


One design decision that, for me, seems to simplify things is to consider the "business system" a type of state machine that records all business events and serves as a "source of truth". If the events are not recorded, they have not occurred from the business perspective. A ledger-type architecture can be useful.

This means that business events or user operations generate state transitions, which eventually are implemented as database transactions. The event log can be stored and inspected.

The end user terms are encoded in the state-transitions. Contract obligations are encoded in the state of the database.

As for calendar issues - operations need to be performed over "real calendars". It might be practical to "materialize" the business calendar into discrete units for this purpose.


For sure; I do think an event based solution working as a source of truth for the billing is the right solution. However it still creates engineering difficulties (making sure you don't ingest the event twice for instance). The ledger-type architecture can definitely work. When we built the system for a fintech, it was actually an event-based architecture connected to a ledger (taking the money out of a wallet). I think the whole process would be: - Ingesting events for usage based feature - Store these events in a database and make sure you make the idempotency right - Aggregate the usage of these events based on most common aggregate functions - Price this usage inside plans or subscriptions - Assign a subscription to a customer - Trigger a webhook (used for invoice/payment) at the end or beginning of the billing period


And that events, as much as possible, are formulated as observations or state-changes rather than operations. Events in the form of "set x=1" or "y=0 was observed" are idempotent, where as "do this with that" is problematic.

In the framework of state-transitions, actions such as "price this with that" are implicit in the state matrix, and shouldn't be called from APIs. Rather, API calls should be used to produce reports which are print-outs of the "current state". Webhooks in this framework are attempted state-transitions. User interactions generate events, and requests for reports (for example invoices), but as little as possible in the form "do this with that".

My opinion currently is that there is little value in sharing code (libraries/frameworks) for these sorts of solutions because the code is trivial, application/language/domain-dependent, and coupled to everything else in the system.

But design patterns, templates, and concise examples are hugely valuable because they help illustrate the complexity and solutions up front.


I was reading this and thought event based system for generating the billable quantities. I am sure billing by api call or data consumption must be done by an event driven system, if my customers are doing a billion requests a month, I probably do not want that data in my database. I would have also thought logging would be your friend in generating reports to reconcile your accounts. Some form of message cache to ensure it has not been processed twice would help.


Nit: there's a lot of noise being made in comments about it being an open source product, but I haven't found the source (maybe I have to speak to an expert to get it?). github.com/getlago has the docs, which is nice (and gives a lot more info than is available on the website about how the product works), but not quite the same.

Also note:

> Our open-core version is forever free. We will introduce paying add-ons in the future, with a consumption-based approach.


Hello! As promised, the repo is available here: https://github.com/getlago/lago ! We also opened our roadmap: https://github.com/getlago/lago/projects/2 You can request and vote for features here: https://community.getlago.com/c/feature-requests/11


It's coming soon, we're in the final steps of QA, but we could not wait to share this post! Thanks for the comments on the documentation, by the way!


Awesome, glad to hear, would love to take a look whenever it's available! Looking at the docs it seems like the current state of the product is much more about subscriptions than usage based billing (e.g. no aggregation or rating). Any plans to add those in?


We have an event-based architecture and offer 'aggregations' at the moment: https://doc.getlago.com/docs/guide/billable-metrics/aggregat... We plan to add more aggregation types, gauge metering as well!

We definitely want to address usage-based pricings, but also all the pricings that are hybrid between usage and subscriptions (ie the vast majority of software pricings). Feel free to reach out if you have more question: anhtho@getlago.com


I spent a lot of time right out of college working on the equivalent of an invoicing system for a pharmacy chain (rx drug pricing, electronic carrier submission, and reconciliation).

Account receivables is also another nightmare. We would get checks that randomly show up at corporate for no reason from insurance carriers but then when the carrier realized their error instead of handling it as a separate process they would deduct the check amount from whatever invoice ( or even across a range of invoices! ) from us they felt like. We literally had a bucket called "magic money" these random deposits went into that we would use to fill in the A/R gaps from insurance carrier insanity. There was no connection between magic money and whatever invoice they decided to short change us on so it was just a hope-for-the-best process.


Hey, I feel your pain. The "hope-for-the-best process" definitely is something happening in many companies... Yet, it's not a fatality anymore. I work with Upflow (YC 20) and it is exactly what we offer to our clients : 1. Making sure they have clear visibility on their AR 2. designing systematized workflows for their customers (as many as needed) 3. Customer payment portals to help their customers pay on time, with no friction. That's here at www.upflow.io


I've implemented a few billing systems and "the nightmares" were never related toengineering but always to specification: If bizdev does not know what they want, no outsourced imlementation will be able to help them.


From your experience, who should specify the billing system? The Product team?

We're building Lago so that the right questions / decisions frameworks are asked during the implementation, so it's like a forcing power embedded in the product.

Our experience is, unless the Product/Biz team has been exposed to billing, they will never specify in a way that is precise enough, so the engineers who implement billing will have to think/assess/decide themselves.


Have someone from accounting/legal work at least 50% in bizdev; don't let them talk to engineering until they have sorted out what they want and, much more important, whether it is legal..

But equally important is to cut back engineering's enthusiasm for abstraction/unification": In my experience it is better to have 2 distinct, tested implementations against one pretty generic data model than trying to coerce both into one. Intellectually one can, but "oh so great billing concept" #3 or #4 will break ones neck.

I don't really think anything about billing is hard, it is just incredibly dull if done the pragmatic way.


'Tax logic' is an oxymoron and a trap for smart technologists to fall into. Tax codes are not logically consistent or clearly defined. They're a bunch of illfitting laws, beauracratic/regulatory guidance and interpretations of judicial decisions.

I tell folks working in this area to accept the illogical nature of it all and to be ready for all sorts of arbitrary last minute changes. As the article points out understanding time plays an important role here too - when do different tax treatments take effect is an important question to answer.


Tax is written by a bunch of people whose background is overwhelmingly from law, not finance, for reasons that have nothing to do with logic, consistency, financial literacy, understanding, sanity, or common sense. Politicians frequently pass laws without understanding what it will do or what its implications are, because they just didn't think that far ahead. It is quite possibly the worst system you could come up with for writing tax law.

That said, there is a logic to most of it, but not in a way that allows you to come up with general heuristics or universal abstractions.


Any kind of business or accounting system tends to be an endless long tail of "can the invoicing system send the invoice in Esperanto using only 16-bit Unicode characters on Tuesdays but only while it is raining and only for customers whose last names end with E and who signed up more than one year ago according to the Chinese calendar?" type features.

That's why attempts at "no-code" systems where biz/accounting people can design their own system are a perennial fad, but as these systems are developed they always just end up evolving into weird domain specific programming languages.

It's also why any sufficiently large company ends up implementing their own in-house accounting system and/or customizing some byzantine monstrosity like SAP which is really just another way of implementing your own.


The #1 reason those systems are moderately successful: When your sales person needs to write the logic for billing customer X, they'll try to make the customer contract more reasonable.

Normally sales people are deal-driven, and if promising the customer "14% off on the first three transactions on every rainy Tuesday, unless Monday was also rainy" gets the deal closed, then that's going in the contract.

No better way to ensure the contracts are reasonable, than making sure the person negotiating the deal feels the pain of billing it


Agreed! this triggers also complex behaviors because sales people want to handle this in their env (Salesforce, hubspot or whatever); You also have to build the logic to update subscriptions and pricing directly from it, in a two-way sync. Things getting crazier ;)


Just wait until you have to do this under government controlled (tariff) environments. Years ago I worked on a tele-conference platform for large international telecom. We built the software to run all aspects of their platform including billing.

Government tariffs covered billing rules for each leg of a call including rates based on caller's local time of day. Converting bridge time to local time via area/country codes(only data available) and time-zones is quite a mess. Add DST calculations to that per locale to reconcile negative duration legs or legs that should be 1.5 hours but show up as .5 hours just makes it cloudier.

If your system repeatedly gets it wrong, you could be in trouble for billing fraud or tariff violation. No joy in that job.


I agree with thesis but there are entire aspects of this that he didn't even touch upon. There is a multiverse of different billing nightmares for engineers.

I worked on a piece of software that bridged billing data from IBM mainframes out to the web. The hoops that had to be jumped through to get data out of the mainframes and the number of hacks and kludges involved was legendary. Likely everyone here has paid many bills in their life on websites that used that software. At one point I knew the vast majority of my personal bills had gone through it. Credit Cards, Utilities, Gas cards, etc.. Stuff everyone had to pay with paper before the web.


Could you expand on what kind of hacks/kludges were required?


Basically the mainframes were almost incapable of any data exchange that was meaningful in the 21st century. Instead of IBM doing something same the market just built software to scrape bills off the mainframes printer output, which you could intercept in electronic form. But then every accounts bill files were different. And sometimes the mainframes moved data around on the page, and so on and so forth.


That gives me flashbacks. I worked on a payment system that had to pull data from third-party service providers that only made it available in the form of downloadable spreadsheets from godawful slow and unreliable JavaScript-heavy web portals. Just getting to the point of downloading the spreadsheets was painful enough, and then the spreadsheets themselves were inconsistent and would sometimes flap back and forth between two different formats for a few weeks as the service provider repeatedly deployed and then rolled back new versions of their software.


Always has been. Marketing and sales create complexity on the front end, devising all kinds of products, plans, discount schemes and related complexity; jurisdiction, taxes, and accounting rules add complexity on the backend. In the middle, measuring what you're billing for is the hook around which all that complexity chaotically orbits.

Decades ago - the computer involved was an honest to goodness System 360 (which is to say, not a 370, Z, or any of the the code compatible 360 successors) - I spent an evening trying to figure out the service charge on my bank account. I couldn't get at all close to the figure on the statement by applying the bank's schedule of charges. I took it into the bank, and got bumped by the teller to a department manager, and thence to a VP (small town; small bank), whom I told, "I will pay the service charge if you can explain to me what I'm being charged for." 2 hours later he gave up, having not gotten any where near an explanation, grinned, and said, "I can fix this." He set some flag on my account, and I never again paid service charges at that bank. Yep - that flag remained set on my account for nearly 20 years, through multiple software and hardware upgrades, until the bank was acquired by a big, expanding regional bank, and I left for a different, entirely local bank.


The marketing team of my previous company created a whole fake price plan that never existed in the code base. Somehow it works to drag acquisition, but it’s a pain in the ** to explain why it’s not as easy as updating webflow ;)


Was I really such a pain? :)


As a co/founder and core architect of multiple PCI level 1 compliant payment processing companies in the last two decades I will add as others have that billing and payments is exactly like anything else which is "difficult". Once you spend enough time in the trenches dealing with the challenging technical issues your experience and creativity leads to the proper solutions that further enhance the system. Everything is just inputs and outputs yet it is what one does in between those "puts" which matters and thus where experience can reduce such challenges.


Why XXXXXXX are a nightmare for engineers:

PEOPLE

On a more serious note though billing can be hugely complicated and I am really glad I do not have to deal with it


Do you have a dedicated team in charge of it?


Absolutely agree with the poster. This is my life at the moment. Just to take money for a small piece of software I built.

Multiple currencies, trial periods, when in the process do you ask for the creditcard, country specific taxes,... And the worst: invoices.

While paddle.com takes care of many of those things - I was shocked after realizing how much work this is going to be.

Please think carefully about those things before you quit your job to work on your "Micro SaaS" or "unicorn startup".


Have you looked at Outseta? https://www.outseta.com/billing and https://www.outseta.com/demo . I haven't used it but it looks great. I'd love to hear what you think.


I haven't but will do. Thanks for the links!


Have also been considering Paddle. Would love to know what you found it's missing?


It's hard to figure out what it can do. Everywhere is their branding. As I understand they just send payment receipts, but no invoices.

I'd like to offer a free trial. While they have support for this, a user needs to enter his credit card right at sign up for trials to work.

So in the end I have to build a hybrid solution where my system gets tied to theirs. Still I don't have full control over all e-mails that are sent to the user. I'm not 100% happy, but better than not charging money.


Agreed! But I do prefer to build it for me rather than someone else’s startup, though


Curious, what are you working on?


Raffi and I are building www.getlago.com : the open-source Stripe Billing alternative


Have you not looked at Stripe? Their product is fairly straight forward to set up


Stripe is fairly straight to set up if you sell products in the US and only accept cards.

For selling a SaaS worldwide, Stripe quickly becomes just as much of a nightmare. Taxes, billing, chargebacks, subscriptions, web hooks, different payment methods, VAT validation, card expiries, legislation, changing laws, SCA flows, and my favourite: SCA iframes that become non-responsive on browsers in a certain language. It is a never ending source of edge cases.

Don't ask me how I know... My biggest mistake with my business was that I vastly underestimated how much work it is to accept payments. In my case, it proved to be more work than building the product itself.


Exactly that. While stripe would tick my needs, it's gonna cause some headaches on the long run. They can calculate the taxes meanwhile, but you have to still take care of e.g. MOSS yourself.


I'm the CTO of a company tackling exactly this problem (www.salesbricks.com) and I'd go one further.

The complexity in billing, invoicing, payments etc are _not_ the stuff that computers or systems do. The complexity is and always will be in the stuff humans do.


Don't spend time, effort and money on things that do not bring business value. Any time I see a company with engineers building something they could buy off the shelf [and the thing they're building is not the product] I know their priorities are whack. I've worked for several companies that had more money than brains. They'd spend years, and millions of dollars in salaries, to build shit they could have bought and implemented in two months. And not only that, but multiple iterations of failing to build said thing, by multiple teams.

Just because a tractor is used for farming doesn't mean a farmer should build her own tractor.


At some companies, it takes more effort to go through the procurement process than to build the thing.


The essence of MVP I guess.

Now, back to the article's topic, the real question is: is there an off-the-shelf billing product available out there?


> Is there an off-the-shelf billing product available out there?

There are lots and lots of OTS billing systems. They might work the way you want or need. Billing is the classic "I wanted wheel, except I wanted a round one" kind of problem. Right now, I'm looking for a system that will let sales invoice whatever they put on paper, and be good a structured, recurring billing that is easy to hook up to my SaaS platform. Oh, and Bob in sales wants to be able to edit the invoices after they are issued...


I rewrote an invoicing system (originally in .NET) in Python and yeah, I totally feel this. All the system did was generate a CSV file that could be input into another system (Quickbooks, iirc). The original .NET version was nonsensical at times, replete with bugs, and I spent a lot of time trying to figure out what the original intent was (there were no docs, of course, and finance often had conflicting/incorrect ideas about what the system should do.) Version 1 had to simply replicate the csv generated by the original system exactly, which required purposefully writing some bugs that I discovered along the way.

But yeah, there were all sorts of nasty complexities -- this was a usage based system, but there were (optional) tiers (pay less for higher tiers), as well as certain time segments that were discounted. And then there were cases where multiple accounts could share a set of data and get a discounted rate and/or only one of the accounts would be billed. And on and on....

One lesson I drew from this experience was that, when discussing things with finance to try to figure out what the requirements were, it really helped to have pictures. I would create these elaborate usage/time graphs, and color in portions to reflect different billing/pricing scenarios. These made it much easier to discuss the current/future behavior of the system. Because saying "really, this is simply the integral of a usage function over time" didn't fly.


This has been my life for the past few months. Billing and dates / times are the worst part of the job.


Have you looked at existing solutions? Why did not they fit?


Not the OP, but anybody who has built a deep integration with any billing provider (inclusive of Stripe) knows there are some bodies buried, although you typically don't discover them until you're late in the game.

90% of your use cases are covered, and with excellent UX and docs to boot. But that sweet 10%!

---

For a concrete example, Stripe has a very nice "subscription" abstraction that makes it easier to operate a SaaS business. They also offer a nice "tax" product, that doesn't do filing for you, but does offer tax computation if you code your products appropriately. Sounds great!

But if you want to sell 2 subscriptions to the same customer, and those 2 subscriptions are fulfilled in 2 distinct taxable regimes (say, you're shipping widgets to NYC once per month and to a vacation home in NH once per quarter), you're screwed because Stripe only allows 1 shipping address for customer for tax purposes, and each subscription references this address when computing tax.

We went through support, which filtered through to engineering, and the eventual answer was "no plan to change this anytime soon", which is difficult to interpret any other way than "good luck". :)

There is no clean or even acceptably-dirty solution to this problem, tragically. So as a business we can't allow customers to transact with us this way. Oh well.

There's another, similar, story around creating draft invoices to give people quotes before checking out. The dashboard API has a function to modify the line items of a draft invoice, but it's not a public-facing API and there's no short-term intention to make it public-facing, so one has to build an ugly workaround.

This effectively means one needs to maintain internal models for subscriptions, products, taxes, invoices, invoice items, products, shipping addresses, on and on. And Stripe becomes a somewhat-dumb processor for these models (although the state machine for subscriptions is admittedly very valuable).

I'd love to "offshore" everything to Stripe, status as the source of truth for data included, but it's not feasible in a surprising frequency of scenarios.

Caveat that every payment provider is like this, Stripe is much better and actually improves over time.


A lot of the time there’s trade-offs that can be made that engineers don’t even consider because they’re not pure. For example, why do you need a one-to-one relationship between your customers and Stripe customers? I am guilty of it myself, where an 80% solution is unacceptable from an engineering perspective but, actually, when balanced against the cost of building out a perfect solution, the business may well find it very acceptable.

One of the most important things I’ve learned about working with third-party systems is that there has to be some amount of surrender to their way of doing things. Essentially, the price of using Stripe is not being able to use invoices as a quote, and that has to be articulated to the business. If that’s not acceptable, then the business should be considering the requirements in aggregate when evaluating a solution, rather than deciding on Stripe and then gradually re-implementing parts of Stripe internally because they aren’t right for the business.

(Obviously that’s easier said than done, and for engineers it’s a fun challenge to make something impossible possible, but it’s almost always a mistake and as engineers it’s our responsibility to articulate the burden of owning workarounds.)


The example you give is a smart one that we did explore!

Effectively, relationship is no longer user -> stripe customer, but user -> shipping_address -> stripe customer.

We decided the cost of implementing and maintaining this solution wasn't worth the benefit to the business. A credit to your later points.

We came up with a solution that allows us to ship one-off things to arbitrary addresses with correct tax calculation, and if a customer really wanted to setup a subscription to a distinct address, we could implement it in the future, hackily, with this primitive.


>One of the most important things I’ve learned about working with third-party systems is that there has to be some amount of surrender to their way of doing things. Essentially, the price of using Stripe is not being able to use invoices as a quote, and that has to be articulated to the business.

We have this same limitation on another payment platform. We just change the concept on our end to "payers" and allow an account to have multiple payers. When the user adds a payment method, they can either select that existing payer or, if they need to change the name or address, they create a new one.

The provider we use had an invoicing product, but it has several misfeatures that just make the product unworkable. We absolutely have to be able to enforce first-in-first-out invoice payments. Users will pay the May invoice before the April invoice and pretend like they are all paid up. I'm not sure why they didn't consider this with a subscription-invoice product... users paying the invoices out of order was a big problem because users saw it as an exploitable vulnerability they could use to get a free month here and there.


> There's another, similar, story around creating draft invoices to give people quotes before checking out. The dashboard API has a function to modify the line items of a draft invoice, but it's not a public-facing API and there's no short-term intention to make it public-facing, so one has to build an ugly workaround.

There is a similar "not a public-facing API" limitation with configuring multiple invoice email addresses for Stripe customers (works in the dashboard, but blocked in the public API, where you can only set a single email per customer).

I'm actually quite surprised Stripe does this, since their initial reason for being was developer-friendly payments. But oh well, maybe I should take a look at an alternative (like Lago) sometime.


We're working on making the quoting process simpler indeed. What we've learnt that the only companies that can afford a proper quoting system are enterprises: they implement Salesforce CPQ (configure price quote) or Zuora. They are mostly sales-driven, so the investment is worth it.

But it does not make sense for PLG companies, who start with self-serve and layer sales afterwards: the investment is not worth it, and it creates 2 different systems: self-serve and sales-driven billings, not great for the user experience (and for the internal processes!)


I think it's easier for billing systems to have 1 subscription for 1 customer. The reason behind this is mainly because you can apply several prices to the same feature belonging to two different subscriptions. Stripe does not work on it because they might consider this as a edge case.

When you look at usage-based billing, even if a billed feature is linked to consumption, it needs to be related to a subscription given the fact that this feature can be priced differently for several plans. Every company has a singular pricing model in mind and it's hard to have an all-in-one model fitting all the needs. Even when we built internally the whole system for a fintech back in 2016, we had some edgecases creating conflict inside the company.


It definitely is an edge case.

But I'd push back on the 1 subscription per customer idea. Consider any physical fulfillment business, where you have an optional annual membership (amazon prime, costco, whatever) and N subscriptions of varying frequencies for physical products.

It doesn't make sense to roll these into a single subscription on the backend (unless the periodicity happens to line up perfectly, and even then there's a heated debate, since you still probably want them to be separate charges).

You can go down the route of creating N "virtual" customers for each "real" customer, but it doesn't really solve the problems, because these N customers have a complex relationship with each other that you need to encode and manage.

Hence, the name of this post :). It's definitely not easy!


This is what I’m doing, but then you don’t get a clean history in Stripe, which is annoying to folks (e.g. to our support team). So, it’s something I’m thinking about changing.


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: