There are a lot of things that are treated as a given in modern software development. Things like continuous deployment, having direct access to the servers where your software runs, being able to easily get instrumentation and logs, and even being able to control configuration. However, there is a startlingly common type of application where you can’t count on any of these things to be true. That type of application is a white label application and it’s the type Will works on every day.
With a white-label app, your clients may tinker with the database, integrate with things you wouldn’t expect, or even come to rely on bugs. They may use the application in ways you never expected, and their business may be entirely reliant on misusing your application. In addition, they may well customize the application itself, changing code in the database, altering CSS, or even changing data that you thought was static. It’s an entirely different set of constraints, and full of nasty surprises for the unprepared.
White-label applications are tricky. They add a lot of constraints onto development that you don’t have in many commercial and in-house software products. That said, they represent a sizable chunk of our industry. It’s important to know what you are getting into before you start working on one. They can be challenging and fun to work on, if you are prepared. A lot of the constaints you are going to deal with in a white-label application apply to varying degrees in many other application types as well.
07:20 What is a white-label application and why would a client want one?
It’s an application for your clients to offer to their clients. This means that you don’t usually have direct access to the end users of your application. This also means that you not only have to support the client’s clients (hereafter “end clients”), the support staff of your client, and your own support staff. This can make many things more difficult.
They want it to look like “their” app, not your app. This means that you have to provide appropriate hooks to allow the client to customize the appearance (and function) of your application. This also means that you have to be careful about what you change in the application.
“These are expensive, both to develop and to modify.”
They probably have additional code for their customizations. This code may be quite extensive, especially when you consider that there may be entire other applications that connect to it. This also means if the database behind the application is on-site, they may have integrated into it. This can include changes to stored procedures, tables, triggers, foreign key constraints, etc.
“You have to future proof the future proofers.”
12:15 How development changes.
“You may not be able to upgrade to the new hotness as quickly as you’d like.”
Many clients will keep old operating systems and browsers around far longer than you would think. You may be expected to support internet explorer 6 and windows XP. This means either not using newer features, or having to deal with things like polyfills. It can also mean having to do a lot of hacks simply to deploy and maintain your code at a client site. This can also mean running on dicey, old hardware with resource constraints.
“From a development standpoint this is why you should make everything private.”
You have to code in a far more defensive manner than you might be used to doing and further into the structure of your application. Because white-label customers are customizing your software, they’ll have a tendency to use/misuse any publicly exposed things you have, whether those are types, API endpoints, or even database structures. As a result, you can’t make nearly as many assumptions about incoming data (you shouldn’t do this anyway), as data may be coming from somewhere you don’t expect. You also have to be careful about how you change validation, as you are no longer the only consumer of your code.
You have to change the way you write error messages. Errors need to be descriptive anyway, but you also need to be careful about changing error messages, as some clients will base their code off of the TEXT of an error. You should somehow return non-string (probably numeric) error codes that are unique for your application. This will give the clients something else to key off of instead of text. You have to be careful about spelling and grammar in error messages as well, because they may appear on the client system if things really go wrong. Bad grammar and spelling will reflect badly on the client.
“One company’s ‘account’ is another company’s ‘customer'”
You have to change the way you deal with strings, dates, and format strings. While sanitizing user input was covered in a previous section, you are likely to also have to allow some customizability for the strings used in your application. Your clients (or their clients even) may want to refer to different things using entirely different terms. This can be especially fun when you have to change the terms within strings, including pluralization rules, internationalization, and different time formats. You’ll also have to deal with all the fun and games that timezones and languages can cause you.
You’ll also have to deal with differences in local law. Data retention and security standards (and the accompanying audit practices) change per jurisdiction. The client (and their clients) will expect this to be done well with your app. In some cases, you can be held liable for screwing up, even if you don’t live in your client’s jurisdiction.
31:40 How testing (should) change.
You have to be far more thorough. You just about have to test the entire application before a release, because an update is an expensive thing. You also need to test the failure modes of different components of the application. You don’t control the whole architecture any more. You also need to test and see what happens when bad data gets placed in your databases, because you may not have as much control over those. You also need to test failure modes around configuration constraints, like what happens when a firewall port is closed, a DNS issue occurs, or when you can’t write to a particular directory or file. You need to test what happens when the database schema (or other data structures) are different than your application expects. If you can’t make it work anyway, you need to provide useful diagnostic error messages so that you don’t cause problems.
“You also have to handle this in a transient case…”
You’ll have to test integration points far more thoroughly based on how your clients use them. You have to test at the component level as well as larger integration tests. This includes regression testing the errors that your application emits to make sure they haven’t been altered. You have to test what happens when multiple instances of your application are deployed. This can include multiple instances in multiple versions, as people screw up during rollouts and not all rollouts can always happen during an outage window. You’ll have to test how individual components fail under load, as different clients will probably have different load profiles.
You may need to provide your test suites to clients for testing in their environment. If it is business critical, your clients should be testing before they roll out. Your test suite should be run on the client environment to check assumptions. You can’t expect all your clients to actually test the software before rolling out, so you’ll need to have a subset of the tests that can be safely run on a production environment to make sure things aren’t broken.
39:55 The Database
“This section here is a big reason why I tell people your DBA hates ORMs”
You need to set expectations early on who can change what in the database. You probably should have some areas that are considered off-limits for client changes, and should check that at install time. You need a zone of no surprises, especialy if you are directly interacting with tables. You may have other areas where you provide sensible defaults, but the clients are allowed to edit. Consider wrapping these in views and stored procedures and calling those so that clients can transparently change these without breaking your application. Direct table access is not a good idea here. This also means creating structures that the clients can use and versioning them appropriately.
You need to assume that the client DBA will do both amazingly smart things and amazing stupid ones that you never planned for. This can include writing bad triggers that cause exceptions to be thrown. It can also include clever performance optimizations. Provide reasonable default database maintenance scripts, but don’t expect all clients to use them properly. Appropriate maintenance should be part of your setup guidance.
“This is something I harp on a lot, everywhere really but database especially.”
You need sensible logging. When an action is intiated from your code to the database, you need to log it. It’s also critical here to set things like the application name (if you are able to do so), so that if the client is logging things or troubleshooting performance, they can tell who is causing the problem. You don’t want to get blamed if it isn’t you and you don’t want to deflect blame if it is.
You need sensible audit trails. If the contents of table are at all important, you need to track who made changes. This includes the account being used to make changes, the initiating application, and potentially the web user that did so. Don’t underestimate how frequently a client DBA will forget a WHERE clause and then blame you.
You need to soft-delete. Don’t allow deletion of sensitive data and don’t do it. Instead, flag the item as deleted and don’t show it to the application. Previous actions are useful for forensics and billing. Deleting a client doesn’t remove the fact that they are obligated to pay their bills.
Big clients will want to shard the database between their clients. Either the database will be too big for all of their clients to be on it or they will have certain clients who need to be separate from the rest. You’ll need to think about this early on in the application design – it’s painful to add this later. There’s also the added fun of how database updates are deployed when there are multiple databases.
“You also need to be careful for when they let their guy in the art dept put in CSS and he injects something dangerous.”
Develco White Label IoT
Develco, a company that makes internet of things products, has a line of white label products. They allow business large and small to put their logo on products such as sensors, alarms, smart plugs, meter interfaces, and gateways. They also put the logo on boxes and inner sleeves.
Tricks of the Trade
There is a time and a place to call someone out at work, even management. Public forums and social media are rarely the place. It may work if you’re trying to get a corporations attention but doesn’t do well if you are outraged at something with your job. If you are upset about a decision that management has made talk to them privately about it or if you need support gather a group of coworkers to address management. Calling them out in public makes you look petty and rarely is it effective. They are more likely to get defensive than to see your point. If your emotions do get the better of you and you have called them out in a public forum then when they say it’s over drop the issue. Not doing so shows a lack of respect, you may be right but you haven’t given them a way to save face.