These are the news items I've curated in my monitoring of the API space that have some relevance to the API definition conversation and I wanted to include in my research. I'm using all of these links to better understand how the space is testing their APIs, going beyond just monitoring and understand the details of each request and response.11 Oct 2017
I was doing some API security research and stumbled across vFeed, a “Correlated Vulnerability and Threat Intelligence Database Wrapper”, providing a JSON API of vulnerabilities from the vFeed database. The approach is a Python API, and not a web API, but I think provides an interesting blueprint for open source APIs. What I found interesting (somewhat) from the vFeed approach was the fact they provide an open source API, and database, but if you want a production version of the database with all the threat intelligence you have to pay for it.
I would say their technical and business approach needs a significant amount of work, but I think there is a workable version of it in there. First, I would create a Python, PHP, Node.js, Java, Go, Ruby version of the API, making sure it is a web API. Next, remove the production restriction on the database, allowing anyone to deploy a working edition, just minus all the threat data. There is a lot of value in there being an open source set of threat intelligence sharing databases and API. Then after that, get smarter about having a variety different free and paid data subscriptions, not just a single database–leverage the API presence.
You could also get smarter about how the database and API enables companies to share their threat data, plugging it into a larger network, making some of it free, and some of it paid–with revenue share all around. There should be a suite of open source threat information sharing databases and APIs, and a federated network of API implementations. Complete with a wealth of open data for folks to tap into and learn from, but also with some revenue generating opportunities throughout the long tail, helping companies fund aspects of their API security operations. Budget shortfalls are a big contributor to security incidents, and some revenue generating activity would be positive.
So, not a perfect model, but enough food for thought to warrant a half-assed blog post like this. Smells like an opportunity for someone out there. Threat information sharing is just one dimension of my API security research where I’m looking to evolve the narrative around how APIs can contribute to security in general. However, there is also an opportunity for enabling the sharing of API related security information, using APIs. Maybe also generating of revenue along the way, helping feed the development of tooling like this, maybe funding individual implementations and threat information nodes, or possibly even fund more storytelling around the concept of API security as well. ;-)
I’ve been playing with Tensor Flow for over a year now, specifically when it comes to working with images and video, but it has been something that has helped me understand what things looks like behind the algorithmic curtain that seems to be part of a growing number of tech marketing strategies right now. Part of this learning is exploring beyond Google’s approach, who is behind Tensor Flow, and understand what is going on at AWS, as well as Azure. I’m stil getting my feet wet learning about what Microsoft is up to with their platform, but I did notice one aspect of the Azure Machine Learning Studio emphasized developers to, “publish, share, monetize” their ML models. While I’m sure there will be a lot of useless vapor ware being sold within this realm, I’m simply seeing it as the next step in API monetization, and specifically the algorithmic evolution of being an API provider.
As the label says in the three ML models for sale in the picture, this is all experimental. Nobody knows what will actually work, or even what the market will bear. However, this is something APIs, and the business of APIs excel at. Making a digital resource available to consumers in a retail, or even wholesale way via marketplaces like Azure and AWS, then playing around with features, pricing, and other elements, until you find the sweet spot. This is how Amazon figured out the whole cloud computing game, and became the leader. It is how Twilio, Stipe and other API as a product companies figured out what developers needed, and what these markets would bear. This will play out in marketplaces like Azure and Google, as well as startup players like Algorithmia–which is where I’ve been cutting my teeth, and learning about ML.
The challenge for ML API entrepreneurs will be helping consumers understand what their models do, or do not do. I see it as an opportunity, because there will be endless amounts of vapor ware, ML voodoo, and smoke and mirrors trying to trick consumers into buying something, as well as endless traps when it comes to keeping them locked in. If you are actually doing something interesting with ML, and it actually provides value in the business world, and you provide clear, concise, no BS language about what it does–you are going to do well. The challenge for you will be getting found in the mountains of crap that is emerging, and differentiating yourself from the smoke and mirrors that we are already seeing so much of. Another challenge you’ll face is navigating the vendor platform course set up by AWS, Google, and Azure as they battle it out for dominance–a game that many of us little guys will have very little power to change or steer.
It is a game that I will keep a close eye on. I’m even pondering publishing a handful of image manipulation models I’ve been working on. IDK. I do not think they are quite ready, and I’m not even entirely sure they are something I want widely used. I’m kind of enjoying using them in my own work, providing me with images I can use in my storytelling. I don’t think the ROI is there yet in the ML API game, and I’ll probably just keep being a bystander, and analyst on the sideline until I see just the right opportunity, or develop just the right model I think will stand out. After seven years of doing API Evangelist I’m pretty good at seeing through the BS, and I’m thinking this experience is going to come in handy in this algorithmic evolution of the API universe, where the magic of AI and ML put so many people under their spell.
I remember when almost all the APIs out there gave us developers access to things we couldn’t ever possibly get on our own. Some of it was about the network effect with the early Amazon and eBay marketplaces, or Flickr and Delicious, and then Twitter and Facebook. Then what really brought it home was going beyond the network effect, and delivering resources that were completely out of our reach like maps of the world around us, (seemingly) infinitely scalable compute and storage, SMS, and credit card payments. In the early days it really seemed like APIs were all about giving us access to something that was out of our reach as startups, or individuals.
While this still does exist, it seems like many APIs have flipped the table and it is all about giving them access to our personal and business data in ways that used to be out of their reach. Machine learning APIs are using parlour tricks to get access to our internal systems and databases. Voice enablement, entertainment, and cameras are gaining access to our homes, what we watch and listen to, and are able to look into the dark corners of our personal lives. Tinder, Facebook, and other platforms know our deep dark secrets, our personal thoughts, and have access to our email and intimate conversations. The API promise seems to have changed along the way, and stopped being about giving us access, and is now about giving them access.
I know it has always been about money, but the early vision of APIs seemed more honest. It seemed more about selling a product or service that people needed, and was more straight up. Now it just seems like APIs are invasive. Being used to infiltrate our professional and business worlds through our mobile phones. It feels like people just want access to us, purely so they can mine us and make more money. You just don’t see many Flickrs, Google Maps, or Amazon EC2s anymore. The new features in mobile devices we carry around, and the ones we install in our home don’t really benefit us in new and amazing ways. They seem to offer just enough to get us to adopt them, and install in our life, so they can get access to yet another data point. Maybe it is just because everything has been done, or maybe it is because it has all been taken over by the money people, looking for the next big thing (for them).
Oh no! Kin is ranting again. No, I’m not. I’m actually feeling pretty grounded in my writing lately, I’m just finding it takes a lot more work to find interesting APIs. I have to sift through many more emails from folks telling me about their exploitative API, before I come across something interesting. I go through 30 vulnerabilities posts in my feeds, before I come across one creative story about something platform is doing. There are 55 posts about ICOs, before I find an interesting investment in a startup doing something that matters. I’m willing to admit that I’m a grumpy API Evangelist most of the time, but I feel really happy, content, and enjoying my research overall. I just feel like the space has lost its way with this big data thing, and are using APIs to become more about infiltrating and extraction, that it is about delivering something that actually gives developers access to something meaningful. I just think we can do better. Something has to give, or this won’t continue to be sustainable much longer.
Amazon Web Services recently updated their billing for EC2 instances to be by the second, which I really like, because I’ll fire up an instance and run for minutes, then shut things down. I’m just looking to process patent downloads, or other intensive workload projects. Beyond just EC2, the rest of Amazon’s platform is still very usage based. Meaning, you get charged for whatever you use, with unit pricing for each resource designed to compliment how it gets put to use. You get charged for the hard costs of compute, storage, and bandwidth, but you also see per message, job, entry, and other types of billing depending on the type of resource being delivered via API.
With this model for doing APIs, I’m wondering why so many API providers still have access plans and tiers. I’ve vented several times that I think service tiers are a legacy of a SaaS way of thinking and does not scale for API consumers. Maybe back when we used a handful of APIs, but the number of APIs I’m using is pushing 50 these days, and I can’t alway justify a monthly subscription to get what I need. I’m looking to just get access to valuable API resources, and get billed for whatever I use. If I don’t use anything for 6 months, I don’t get billed for anything. Also, I want to be able to run large jobs which consume intense amounts of resources without hitting tier and other limits–just charge me for what I use. If I have a $1,000.00 to spend today, let me spend it. Don’t make me jump through hoops.
I know the answer to my question regarding why so many API startups do this. It is because the resources being provided via the API isn’t the product, us API consumers are. They are looking to ensure a certain level of headcount, monthly, and annual subscribers, so that they can sell us to their investors, and ultimately whoever purchases us like cattle in the end. I’m sure there are other reasons for having pricing tiers, but this is still the primary reason we see SaaS based pricing continue to be so pervasive in an API driven world. For me, if an API provider has tiered pricing, I’ll almost always go look elsewhere. I just can’t manage 40 or 50 subscriptions, and if the number of APIs keep growing, I can’t handle 100-200 subscriptions. I just need to pay for the resources my business needs, and nothing more. I sure don’t have time to be a product in your startup cattle auction.
Pay as you go, usage based pricing is one way the cloud giants will suffocate out the small startups in the future. While startups are trying to court investors, and their acquirers, the cloud giants will just offer a competing service, drop the price to run competitors off, then once the coast is clear, raise prices back up to an profitable state. To compete, more API providers will have to go with a utility based pricing strategy, while also offering wholesale versions of their APIs in the marketplaces for AWS, Azure, and Google. I can’t help think that things might have been different if the scales didn’t tip so hard towards all of us API consumers being the product, and API providers focused more on running businesses, and catering to our real world business needs. Oh well, we got the world we have, I’ll just keep plowing forward.
I have been having more conversations with federal agencies as part of my work with my Skylight partners about API related microconsulting. One recent conversation, which I won’t mention the agency, because I haven’t gotten approval, involved bug bounties on top of an API they are rolling out. The agency isn’t looking for the regular technology procurement lifecycle around this project, they are just looking for a little bit of research and consulting to help ensure they are on the right track when it comes to hardening their API approach.
Micro consulting like this will usually not exceed $5,000.00 USD, and will always be a short term commitment. From my vantage point micro consulting will always be API related, and in this particular case involves studying how other API providers in the private sector are leveraging bug bounties to help harden their APIs either before they go public, or afterwards in an ongoing fashion. After I do the research I will be taking this work back to my team of consultants at Skylight, and we’ll put together formal report and presentation that we will bring back to the federal government agency to put into motion.
This approach to doing APIs in the federal government (or any government) is a win-win. It fits with my approach to doing research at API Evangelist, and it provides API expertise for federal agencies in small, affordable, and bite-size chunks. Government agencies do not have to wait months, or years, and spend massive amounts of money to gain access to API expertise. For Skylight, it gets our foot in the door within government, and helps demonstrate the expertise we bring to the table. Something that will almost always turn into additional micro procurement relationships, as well as potentially larger scale, ongoing project relationships.
Personally, I like my API consulting just like my APIs, small, and doing one thing well. I don’t like consulting contracts that try to do too much. I also like getting paid in chunks that can usually be put on the corporate credit card, and avoid too much purchase order, vendor system, 30, 60, and 90 day wrangling–or worse, not getting paid at all. You’ll hear me beating the micro consulting, and micro procurement drum a lot more in coming months. I’m going to be working to educate more government agencies, and even the enterprise of the potential when it comes to API related content creation, storytelling, training, and research. I’m predicting it will have the same effect as APIs are having on how companies, organizations, institutions, and government agencies are doing business in the digital economy.
API management was the first area of my research I started tracking on in 2010, and has been the seed for the 85+ areas of the API lifecycle I’m tracking on in 2017. It was a necessary vehicle for the API sector to move more mainstream, but in 2017 I’m feeling the concept is just too large, and the business of APIs has evolved enough that we should be focusing in on each aspect of API management on its own, and retire the concept entirely. I feel like at this point it will continue to confuse, and be abused, and that we can get more precise in what we are trying to accomplish, and better serve our customers along the way.
The main concepts of API management at play have historically been about authentication, service composition, logging, analytics, and billing. There are plenty of other elements that have often been lumped in there like portal, documentation, support, and other aspects, but securing, tracking, and generating revenue from a variety of APIs, and consumers has been center stage. I’d say that some of the positive aspects of the maturing and evolution of API manage include more of a focus on authentication, as well as the awareness introduced by logging and analytics. I’d say some areas that worry me is that security discussions often stop with API management, and we don’t seem to be having evolved conversations around service conversation, billing, and monetization of our API resources. You rarely see these things discussed when we talk about GraphQL, gRPC, evented architecture, data streaming, and other hot topics in the API sector.
I feel like the technology of APIs conversations have outpaced the business of APIs conversations as API management matured and moved forward. Advancements in logging, analytics, and reporting have definitely advanced, but understanding the value generated by providing different services to different consumers, seeing the cost associated with operations, and the value generated, then charging or even paying consumers involved in that value generation in real-time, seems to be being lost. We are getting better and the tech of making our digital bits more accessible, and moving them around, but we seemed to be losing the thread about quantifying the value, and associating revenue with it in real-time. I see this aspect of API management still occurring, I’m just not seeing the conversations around it move forward as fast as the other areas of API management.
API monetization and plans are two separate area of my research, and are something I’ll keep talking about. Alongside authentication, logging, analysis, and security. I think the reason we don’t hear more stories about API service composition and monetization is that a) companies see this as their secret sauce, and b) there aren’t service providers delivering in these areas exclusively, adding to the conversation. How to rate limit, craft API plans, set pricing at the service and tier levels are some of the most common questions I get. Partly because there isn’t enough conversation and resources to help people navigate, but also because there is insecurity, and skewed views of intellectual property and secret sauce. People in the API sector suck at sharing anything they view is their secret sauce, and with no service providers dedicated to API monetization, nobody is pumping the story machine (beyond me).
I’m feeling like I might be winding down my focus on API management, and focus in on the specific aspects of API management. I’ve been working on my API management guide over the summer, but I’m thinking I’ll abandon it. I might just focus on the specific aspects of conducting API management. IDK. Maybe I’ll still provide a 100K view for people, while introducing separate, much deeper looks at the elements that make up API management. I still have to worry about onboarding the folks who haven’t been around in the sector for the last ten years, and help them learn everything we all have learned along the way. I’m just feeling like the concept is a little dated, and is something that can start working against us in some of the conversations we are having about our API operations, where some important elements like security, and monetization can fall through the cracks.
People love to point out that APIs are unreliable. You can’t depend on them. They go away at any point, and they just aren’t something you want to be building your business on top of. When in reality APIs aren’t reliable it is the business, people, and investment behind them. The reality of the startup game is that us API consumers aren’t actually the customer, we are just numbers in a larger game where startup founders and their investors are looking for enterprise customers to purchase their startup getting the desired exit. The API is just about attracting consumers, who will do the legwork to bring in users, adding to the value of a company.
As the startup funding landscape continues to dry up, shift, and evolve towards more riskier and volatile versions of investment like ICOs, things are only going to get worse. Of course, few conversation will place the blame on the people and companies behind, but APIs will continue to be the scapegoat for the instability. It works just like the robots coming for your jobs. You never hear that rich people who own companies, that are making decisions to replace workers with robots are coming for your jobs. Its the robots. Technology in many forms makes for a great blame shield, absorbing the responsibility for the volatility, instability, and scams that are going on across the landscape.
In reality, nothing much changes for us API consumers. You need to get to know your API providers, well as the company and people behind them. Study their approach to operating their API. Do they communicate? Do they have proper support? Do they communicate their uptime status? What type of funding is propping them up, and the shape of their business model use. Make sure you always have a plan B if you can, and do not trust that ANY API will be around forever. If possible, come up with failover plans, and run drills with your team regarding what will happen when APIs fail, become unreliable, or go away entirely. Ultimately it is up to you to determine the impact unreliable APIs will have on your APIs. We can blame API providers, startups, and VCs as much as we want, but it is entirely up to us to help stabilize the API space for our users.
Investment, fundraising, and chasing exits will be the #1 area of instability for the API space. After that it will be the network, ranging from network neutrality going away to lack of investment in capacity and security. While the growth in the number of companies, organizations, institutions, and government agencies doing APIs will keep things on forward trajectory, the reputation of APIs will continue to take a hit when it comes to the stories people tell about who is to blame. APIs are the frontline for almost everything occurring online these days, and this will make it the first place everyone will be pointing the finger. It is up to us in the API community to keep telling stories putting the blame where it is deserved. If we are not telling stories, the other side will, and they’ll be firing up the pundits, continuing to let irresponsible API startups get away with their bullshit games.
Note: If my writing is a little dark this week, here is a little explainer–don’t worry, things will back to normal at API Evangelist soon.
I get regular emails from folks telling me how much they love what I do, then asking me to work for free. I’m totally happy for folks to inform me about their company, products, case studies, and other API goings on with their company. This is the bread and butter for my storytelling on API Evangelist–please keep it coming. However, the folks who ask me to work for free, what are you thinking? Where does this ethic come from in the tech sector, where folks expect you to work for their startup for free? It is something that has really gotten to ridiculous levels.
Join our webinar. Write a story for our blog. Contribute to a white paper. Join our podcast. Speak at our event. Teach a workshop. Come visit our company. Talk about our products and services. Help us craft our strategy. Do some research on this subject so we can use it. Tell us what we are doing wrong. Download, install, and play with our tool, and provide us with feedback. Take this 148 question survey that will only take you 45 minutes. Jump on the phone with us so we can pick your brain. Post our infographic to your site. Share this out via your Twitter account. Vote us up on Hacker News. Commit to our open source project. Help us create an API definition. Can you review this project specification. On, and on, and on. Oh, and hey I just wanted to email you again, because you didn’t respond to the last two emails!
Don’t get me wrong. If you know me, and we have history, feel free to ask me for favors, but remember, I do not have a job. I’m happy to help my friends, and love participating in podcasts with smart people, and reviewing my friends writing, because they scratch my back. They throw me bones, and have a history of given me exposure, and paying me for actual work. However, if we do not know each other, and have never talked, why on earth would you be asking me to work for free? Why would I want to take my brand that I’ve invested in building for seven years, and transfer that value to you for nothing? I don’t know you, your company, or anything about the value you deliver. I have to make a living just like you, and I can’t be working for free, or there would be nothing of value for both of us.
I know many of you are excited by the seemingly constant forward motion of the tech space, and that you might not realize that you are being rude. You may not be able to see it because you are in the middle of it, but much of startup culture is built on exploitation of others, and just because you are willing to be exploited by your investors, and those around you doesn’t mean you should expect others to fall in line. I’m a professional, and I’m happy to share my knowledge of the space, but it means you need to meet me halfway, and strike a business deal with me. This means emailing, DM’ing, and contacting me in a way that sets that table, not just expecting I will work for free. I appreciate your excitement, but please respect my time, and all the work I’ve put into my business, and I’ll do the same for you.
Note: If my writing is a little dark this week, here is a little explainer–don’t worry, things will back to normal at API Evangelist soon.
I’m really honored that some of my partners are kind enough to offer me a piece of the action in their companies, in exchange for what I do. I really am. However, going forward I’m going to have to decline any stock options in exchange for work, or advising, because it really doesn’t pencil out for me. I know it is the currency you are working with, getting investment in exchange for options, and trying to get knowledge, talent, and other forms of investment in exchange as well. I trust it will work out in your favor, but from my vantage point, there really is no upside in the game.
I regularly receive accusations of having and agenda because of real or perceived interest in companies, so with this hit on my brand, and the lack of return from the historic stock options I’ve had historically, it just doesn’t work out. My last advisor options netted me $319, for about 60 hours of work on, and travel costs out of my own pocket. I’m sure if I had more capital investment, and a bigger piece of the action, I’d fare better, but for the stake I get thrown–it just doesn’t do it for me. Plus, I have a manila folder in the filing cabinet of more significant shares that aren’t worth the paper they are printed on, so stock options really doesn’t float my boat in the first place.
Please don’t let this hurt your feelings. I know you are heavily invested in the value of your options, but I’m just not there. I’m happy to find other arrangements to make things happen, but with the way that stock options get wielded against me, and used to potentially discredit what I’m doing, it just isn’t worth anymore. As it stands today I only have stake in a single company–Skylight Digital (5%). I have NO stake in any API related company, and I am going to keep it that way. I’m not 100% against an equity stake, but the risk of low stake investments, and the high risk of damage to my brand leaves me extremely critical about any conversation in this area.
It is another fascinating area of the startup game in my view, where people like punching down, instead of up when it comes to questioning the ethics and credibility folks in the API space. People be all questioning my ethics, but afraid to ever go after startups, or investors, because it might hurt their chances to be invited into the club some day! It’s a crazy financing reality in the API space, where up is down, and down is up, and I’m just beating that API Evangelist drum. #onward
Note: If my writing is a little dark this week, here is a little explainer–don’t worry, things will back to normal at API Evangelist soon.
I am the first to admit that I suck at the money game. I just don’t care. Don’t get me wrong, I’ve made a significant amount of money in my career, and command a phat six figure salary when I’ve done the job thingy, and I don’t have a problem asking for a decent rate when I’m consulting. It is just that I don’t care about climbing the money ladder, because I realized in my early 20s that it was never enough. I’ve had business partners run off with hundreds of thousands of dollars in cash, screw me out of the equity in multiple companies, and I’ve experienced what people will do to get at that next rung of the ladder. I quickly saw that you are never happier with each rung you climb, and often times you end up much unhappier the higher up you go–which is why I stay where I am at.
If you have been one of my partners you know I suck at invoicing regularly. Honestly, I don’t think about it until I have to. Which regularly I forget an entire month. Unless I’m looking looking at eviction, or have something I need to purchase, I’m more obsessed with working. I love what I do (mostly), and if someone would just drop money in my account each month, I just keep beating the API Evangelist drum, and never look to make beyond what it takes to keep me going each month. Being independent is a hustle, and it takes the support of my partners (thanks 3Scale, Runscope, Restlet, and Tyk) to make this work, along with some side consulting hustle to fill in the gaps. I mostly prefer the low monthly sponsorship money because the consulting hustle is a chore, and chasing down the money can be a full time job. I’d say 1/3 of the time its about getting plugged into corporate system(s), 1/3 of the time smooth sailing, and 1/3 of the time I’m getting screwed.
Honestly, the companies who want an NDA, contract, and all the trimmings are the ones I’ll usually walk away from. When I have a contract, I almost always get screwed over. I have more unpaid contracts from more well known corporations and startups than I can shake a stick at. I’m amazed at the folks who talk me up, tell me how much they love what I do, and want my services…then get what they need, don’t pay their invoice, and go radio silent. I also get many folks who tell me how their API budget wasn’t as big as they had hoped this quarter, and my fees would be such a big hit. Another I hear regularly is startups waiting for their investment, and at some point they’ll kick me down when they get theirs. The space is full of excuses regarding how hard-done-by everyone is with their operating budgets, runway, and what not. Yeah, I know what you mean–your hard times means I won’t make rent this month. I get it. I do.
This is all a game. The only difference is I’m admitting it, and you aren’t. I feel like we are all in this together, trying to build community, and you are secretively in it for yourself, and working it from every angle. Don’t worry, I know not all of you are like this, and when I’m done with API rant week, I’ll showcase some of the heroes in my world (you know who you are). I just wanted to take the time to shame some of you for the greedy, inconsiderate individuals you are, playing a nice game on the front, but making sure you always get yours. The differences is that you’ll have a rough time in the next meeting, and I’ll be out of my apartment, unable to make it to my next consulting gig, or just unable to meet my wider personal and professional obligations. But it’s cool, I’m sure you got this all figured out, and you will sleep well at night. It’s just business.
You are in a sweet spot. You got a fat six figure job in the coolest department of your company, building out your API platform. You have a decent budget (never as much as you want) to throw hackathons, run Google and Twitter ads, and you can buy schwag to give away at your events. Sure there is a lot of pressure to deliver, but you are doing pretty well. All you gotta do is convince 3rd party developers to do thing with your companies APIs, develop web, mobile, voice, and other applications that generate buzz and deliver the return on investment your bosses are looking for.
It is all about you and your team. Let’s get to work growth hacking! Attract as may new users as we can, and convince them to build as much as we possibly can. Let’s get them to develop SDKs, write articles for us on their blog, speak at our events, favorite things on hacker news, and whatever activities that we can. Your objective is to extract as much value from your API operations as you possibly can, and give nothing back. Expect developers to work for free. Expect your hackathons attendees to come up with the next great idea, build it, and hand it over to you for very little in return. This isn’t a partnership, this is an API ecosystem, and your team is determined to win at all costs.
Your API isn’t a two-way street. All roads lead to your success, and your bosses getting what they want. You don’t care that 3rd party developers should be compensated, or that they have any rights to their intellectual property. The 5% of them that successfully build applications, we will offer them a job in exchange for it, or we’ll just replicate it internally, decrease their rate limits, and increase their error rates so that they can’t compete. Sure you want people to still feel inspired, but not enough so that they’ll ever be able to sustain their applications–the only sustainable application around here will be owned by the platform. After all, this is all just business–it is nothing personal.
I’ve been looking at my human services API work through a microservices lens, triggered by the deployment of a reduced functionality version of the human services implementation I was working on this week. I’m thinking a lot about the technical side of decoupling services using APIs, but I want to also take a moment and think about the business side of decomposing services, while also making sure they are deployed in a way that meets both the provider and consumer side of the business equation. My human services microservice implementation is in the public service, which is a space where the business conversation often seems to disappear behind closed doors, but in reality needs to be front and center with each investment (commit) made into any service.
Let’s take a moment to look at the monetization strategy and operational plan for my human service microservice. Yes, a public data microservice should have a monetization strategy and plan for operating and remaining sustainable. The goals for this type of microservice will be radically different than it would be for a commercial microservice, but it should have one all the same.
- Monetization - How am I evaluating the investment into this project alongside any value that is generated, which I can potentially capture or exchange some value for some money to keep going.
- Acquisition - What did it take for me to acquire the data and skills necessary to make the deployment possible.
- Development - What time was invested in setting up the platform, developing the schema, data, definitions, code, and visual elements.
- Operations - What does it take to operate the service? Maintain it, answer questions, provide support, and other realities of providing an online service today.
- Direct Value - What are the direct benefits of having this service available to people looking for human services, or organizations looking to provide human services.
- Indirect Value - What are the indirect benefits of having this service available, like increased conversation around human services, or maybe traffic and awareness of the Open Referral organization.
- Partners - What partnership opportunities are actively being sought out, or will be opened up by having this service available.
- Reporting - What type of reporting is necessary to operate and monetize this service, from tracking page views to understanding who is integrated with the data, and consuming data via APIs, or possibly through continuous integration of the Github repository.
- Plan - What is my plan for making this service available to the public, partners, or maybe internally use across my own projects.
- Elements - My human services location API is designed to be publicly available, forkable, and reusable by anyone.
- Limits - There are no limits on how each human services microservice can be used, or forked and reused. Ideally, any implementation provides attribution, and acknowledges the source of framework or data, but there really are no rules.
- Metrics - I am measuring unique page views on each page, including of the machine readable YAML, JSON, and other formats. I’m also tracking on stars, forks, and commits for each service.
- Commercial - Providing a clear track for commercial vendors to understand that the project needs their support, and can be improved upon, and evolved through their underwriting and support.
I need to have a coherent snapshot of what I’ve invested into each of my service. I need to have a base plan for how I will be executing the business side of this service–even if it just making something available for free. There are two dimensions to this conversation: 1) My view as the creator of this service 2) The view of folks who fork and implement as a service. Both dimensions should have a monetization snapshot, and a plan for executing within this business snapshot. I need all of this decoupled from any other service I am offering, but ideally they all use a common set of reusable patterns, just like the technical aspects of my microservices effort.
Just like needing the compute, database, DNS, and other technical layers to be stable and scalable across my microservices, I need the costs associated with these elements predictable and affordable. I need to know what it costs to define, design, deploy, manage, and deprecate my services. I need to know the best path forward for making them public, keeping them private, and being honest with commercial partners about the value that is being generated, both directly and indirectly. I need a way to report my costs, and revenue across hundreds or thousands of these services. I need to be able to scale this both horizontally across many services, but also vertically for single services which get deployed over and over, reused and continuously integrate wherever they are needed using Github.
I’ll keep applying this model across my human services project. I’m thinking that I’m going to be developing a whole buffet of human service microservices that run a 100% on Github like this, but I am also playing around with other varietals that run on other popular cloud platforms as well. I’m not one to subscribe to any particular API dogma, I’m just looking to explore what is possible, and do all of it as cheaply as I possibly scan, growing the number of projects I’m able to tackle. Of course, none of this would be possible without my partners funding my work, and helping me connect the technical and business aspects of doing API Evangelist.
I was reading about the difficulties the City of New York was having when it comes to migrating off of the Palantir platfom, while also reading about the latest cybersecurity drama involving ransomware. I’m spending a lot of time studying cybersecurity lately, partly because they involve APIs, but mostly because it is something that is impacting every aspect of our lives, including our democracy, education, and healthcare. One thing I notice on the cybersecurity stage, is that everything is a much more extreme, intense, representation of what is going on in the mainstream tech industry.
Ransomware is software that gets installed on your desktop or servers and locks up all your data until you pay the software developer (implementor) a ransom. Ransomware is just a much faster moving version of what many of us in the software industry call vendor lock-in. This is what you are seeing with Palantir, and the City of New York. What tech companies do is get you to install their software on your desktop or servers, or convince you to upload all your data into the cloud, and use their software. This is business 101 in the tech industry. You either develop cloud-based software, something that runs on-premise, or you are a mix of both. Ideally, your customers become dependent on you, and they keep paying your monthly, quarterly, or annual subscriptions (cough cough ransom).
Here is where the crafty API part of the scheme comes in. Software providers can also make APIs that allow your desktop and server to integrate with their cloud solutions, allowing for much deeper integration of data, content, and algorithms. The presence of APIs SHOULD also mean that you can more easily get your data, content, and algorithms back, or have kept in sync the whole time, so that when you are ready to move on, you don’t have a problem getting your data and content back. The problem is, that APIs “CAN” enable this, but in many situations providers do not actually give you complete access to your data, content, or algorithms via API, and enable the true data portability and sync features you need to continue doing business without them.
This is vendor lock-in. It is a much friendlier, slower moving version of ransomware. As a software vendor you want your software to be baked in to a customers operations so they are dependent on you. How aggressively you pursue this, and how much you limit data portability, and interoperability, dictates whether you are just doing business as usual, or engaging in vendor lock-in. One thing I’m hopeful for in all of this, are the vendors who see transparency, observability, interoperability, and portability of not just the technical, but also the business and politics of delivering technology as a legitimate competitive advantage. This means that they will always be able to out maneuver, and stay ahead of software vendors who practice vendor lock-in and ransomware, whether of the slow or fast moving variety.
The General Services Administration(GSA) has an API strategy, which describes “GSA’s future direction for agencywide API management including design, development, architecture, operations, and support, and security.” Ok, let’s pause there. I want to point out that this isn’t just an API design guide. That is a portion of it, but it also touches on some of the most obvious (deployment), and the most critical aspects (security) of API operation–important stuff.
The objectives for the GSA crafting an API strategy are:
* Harness API management to maximize customer value and technical efficiencies. * Adopt industry best practices with API design, development, and management. * Consider opportunities for revenue through API offerings.
I always enjoy seeing federal agencies talk about efficiencies and best practices, but it gives me hope that all of this might actually work when I see federal agencies actually “considering opportunities for revenue through API offerings”. Especially at the GSA, who provides technical guidance to the other 400+ federal agencies, as well as at the state and municipal level. I am not excited about our government charging money for digital resources, I am excited about our government exploring how it will generate revenue to sustain itself in the future.
I know there are a lot of open data advocates who can’t wrap their mind around this, but this is how the government will generate needed tax revenue to operate in the future–using APIs. Our government currently generates a portion of its revenue from the sale of physical resources like timber, mining, and agriculture, why should things be different when it comes to taxing and regulating digital resources being made available via the web, mobile, and device applications. While there is still lots to figure out on this front, I am glad to see the GSA putting some thought into the role APIs will play in the future of funding the government services we all depend on each day.
I was having a discussion with an investor today about the potential of algorithmic-centered API marketplaces. I’m not talking about API marketplaces like Mashape, I’m more talking about ML API marketplaces like Algorithmia. This conversation spans multiple areas of my API lifecycle research, so I wanted to explore my thoughts on the subject some more.
I really do not get excited about API marketplaces when you think just about API discovery–how do I find an API? We need solutions in this area, but I feel good implementations will immediately move from useful to commodity, with companies like Amazon already pushing this towards a reality.
There are a handful of key factors for determining who ultimately wins the API Machine Learning (ML) marketplace game:
- Always Modular - Everything has to be decoupled and deliver micro value. Vendors will be tempted to build in dependency and emphasize relationships and partnerships, but the smaller and more modular will always win out.
- Easy Multi-Cloud - Whatever is available in a marketplace has to be available on all major platforms. Even if the marketplace is AWS, each unit of compute has to be transferrable to Google or Azure cloud without ANY friction.
- Enterprise Ready - The biggest failure of API marketplaces has always been being public. On-premise and private cloud API ML marketplaces will always be more successful that their public counterparts. The marketplace that caters to the enterprise will do well.
- Financial Engine - The key to markets are their financial engines. This is one area AWS is way ahead of the game, with their approach to monetizing digital bits, and their sophisticated market creating pricing calculators for estimating and predicting costs gives them a significant advantage. Whichever marketplaces allows for innovation at the financial engine level will win.
- Definition Driven - Marketplaces of the future will have to be definition driven. Everything has to have a YAML or JSON definition, from the API interface, and schema defining inputs and outputs, to the pricing, licensing, TOS, and SLA. The technology, business, and politics of the marketplace needs to be defined in a machine-readable way that can be measured, exchanged, and syndicated as needed.
Google has inroads into this realm with their GSuite Marketplace, and Play Marketplaces, but it feels more fragmented than Azure and AWS approaches. None of them are as far along as Algorithmia when it comes to specifically ML focused APIs. In coming months I will invest more time into mapping out what is available via marketplaces, trying to better understand their contents–whether application, SaaS, and data, content, or algorithmic API.
I feel like many marketplace conversations often get lost in the discovery layer. In my opinion, there are many other contributing factors beyond just finding things. I talked about the retail and wholesale economics of Algorithmia’s approach back in January, and I continue to think the economic engine will be one of the biggest factors in any API ML marketplace success–how it allows marketplace vendors to understand, experiment, and scale the revenue part of things without giving up to big of a slice of the pie.
Beyond revenue, the modularity and portability will be equally important as the financial engine, providing vital relief valves for some of the classic silo and walled garden effects we’ve seen the impact the success of previous marketplace efforts. I’ll keep studying the approach of smaller providers like Algorithmia, as well as those of the cloud giants, and see where all of this goes. It is natural to default to AWS lead when it comes to the cloud, but I’m continually impressed with what I’m seeing out of Azure, as well as feel that Google has a significant advantage when it comes to TensorFlow, as well as their overall public API experience–we will see.
I am going through my entire infrastructure lately, quantifying the products and services that API Evangelist offers, and the partnerships that make everything go round. As I do in my work as the API Evangelist, I'm looking to work through my thoughts here on the blog, and this week I have an interesting topic on the workbench--the API Evangelist remarketing tags.
According to Google, remarketing tags are: "To show ads to people who have visited your desktop or mobile website, add the remarketing tag to your website. The tag is a short snippet of code that adds your website visitors to remarketing lists; you can then target these lists with your ads. If your website has a Google Analytics tag, you can use this tag instead and skip adding the AdWords remarketing tag."
Over the last couple of years, I've had 3Scale's remarketing tags on my site. 3Scale has been my number one supporter for the last three years, and API Evangelist wouldn't even be a thing without them. They have provided me with financial support each month for a variety of reasons. We've never fully itemized what is in exchange for this support, primarily it has been to just invest in me being API Evangelist and help 3Scale be successful, but one of the things on the table for the last couple of years has been that I published the 3Scale remarketing tags on my network of sites.
So, if you visited API Evangelist in the last three years, it is likely you saw 3Scale ads wherever you went on the web. As I'm evaluating all the products and services I offer, quantify my partnerships, and identify the value that API Evangelist brings to the table, I find myself thinking deeply about this practice. Should I be selling my visitors data like this? If you visit API Evangelist, should I allow my partners to target you with advertising on other sites you visit? As I think about the challenges I face in accepting money from startups, I'm forced to look at myself in the mirror as I think about selling remarketing tag access on my website(s).
Ok, I take the high road. I don't sell remarketing tags to my partners. I even take the Google tracking off my website. I'm happy with tracking my traffic at the DNS level, I do not need to do this. Except for the fact that it brings in revenue, I need to pay my server bills, rent, and everything else that keeps me doing API Evangelist. It isn't easy making a living being independent in the tech space, and I need every little bit of revenue I possibly can. It's not an insignificant amount of money either. I make a couple thousand dollars a month this way, as the traffic to my website is pretty high quality, at least in the eyes of an API service provider looking to sell you their warez.
I am still thinking on this one. I need to make a decision this month regarding what I am going to do with the remarketing tags for API Evangelist. Do I want to sell them entirely to one partner? Do I want to sell individual ones for each area of my research, like design, deployment, management, monitoring, and the other 70 areas of the API lifecycle I track on? IDK. It is a shitty place to be, with so few options for monetization of my work, beyond just the current advertising industrial complex (AIC)™. I'd love to hear your thoughts, either publicly or privately. It's a tough decision for me because if I choose to not do it, I risk API Evangelist going away, and me having to go get a job. *shudder* *shudder again* I'm exploring other ways of generating revenue, things like creating classes, and other content I can sell access to, so hopefully I can just walk away from remarketing tags without it killing me off. IDK WTF
It is a hustle to do API Evangelist. I've been lucky to have the support of 3Scale since 2013, without them API Evangelist would not have survived. I'm also thankful for the community stepping up last year to keep the site up and running, keeping it community focused thing, and not just yet another vendor mouthpiece. I make my money providing four ad slots on the site, by selling guides and white papers, and by consulting and creating content for others. It is a hustle that I enjoy much more than having a regular job, even though it is often more precarious, and unpredictable regarding what the future might hold.
Taking money from companies always creates a paradox for me. People read my stories because they tend to be vendor neutral and focus on ideas, and usable API-centric concepts. While I do write about specific companies, products, services, and tooling, I primarily try to talk about the solutions they provide, the ideas and stories behind them, steering clear of just being a cheerleader for specific vendor solutions. It's hard, and something I'm not always successful at, but I have primarily defined my brand by sticking to this theory.
This approach is at odds with what most people want to give me money for. 3Scale has long supported me and invested in me being me, doing what I do--which is rare. Most companies just want me to write about them, even if they understand the API Evangelist brand. They are giving me money, and in exchange, I should write about them, and their products and services. They often have no regard to the fact that this will hurt my brand, and run my readers off, falling short of actually achieving what they are wanting to achieve. I get the desire to advertise and market your warez, but the ability for companies to be their own worst enemy in this process is always fascinating to me.
I get regular waves of folks who want to give me money. I'm talking with and fending off a couple at the moment. So I wanted to think through this paradox (again), and talk about it out loud. It just doesn't make sense for me to take money from a company to write blog posts, white papers, and guides about their products and services. However, I feel like I can take money from companies to write blog posts, white papers, and guides about an industry or topic related to the problems they provide solutions for, or possesses a significant audience that might be interested in their products and services. I consider this underwriting and sponsorship of my API Evangelist research, where they receive branding, references, and other exposure opportunities along the way.
Ideally, all branding, reference, and exposure elements are measurable through the tracking of impressions and links. What was the reach? What is the scope of exposure? It is difficult to sustain any relationship without measuring success and both parties are unable to articulate and justify the financial underwriting and support. In some cases, I get paid a finder's fee for referrals, but this can be very difficult to track on and validate--something that requires a company to be pretty ethical, and truly invested in partnerships with smaller brands like me. I prefer to rely on this, as opposed to any sort of affiliate or reseller style tracking systems. I like companies that ask their customers, "how did you learn about us?", as they tend to actually care about their employees, partners, other stakeholders at a higher level.
Sometimes I take an advisor, or even technology investor role in startups, taking on a larger stake in outcomes, but these are something that is very rare. I have a manila file folder filled with stock options, and stakes I have in companies that never reached an exit, or when they did I was written out of things--when I do get a check from startup founders, I'm always floored. This does happen, but is truly a rare occurrence. I wish there were more decoupled, plug and play opportunities to invest in startups as an advisor, researcher, analyst, and storyteller, but alas the system isn't really set up for this type of thinking--people prefer the big money plays, over smaller, more meaningful contributions--it's about the money man.
Anyways, every time I visit this conversation in my head I come back to the same place. I'm happy to take money from companies to generate short form and long form content about an industry or topic. If there are finder fees for referrals, great! I leave it up to you to track on and come back to me with the details, and any specific event--while I will stay focused on what I do best, the thing you are investing in me to do. I'm mildly interested in opportunities to become more invested in your companies overall success, honestly, I just don't trust that the system will deliver in this area, and is more often just looking to extract value from me. I have seen too much in my 30-year career. However, I always welcome folks who want to prove me wrong! ;-)
In the end, my mission stays the same. I'm interested in studying where the API space has been, where it is at, and where it might be going. Then, I am interested in telling stories from what I learn in this process. If you want to invest in any of this, let me know. I could use your help.
I get why SaaS, and API providers offer a handful of pricing plans and tiers for their platforms, but it isn't something I personally care for as an API consumer. I've studied thousands of plans and pricing for API providers, and have to regularly navigate 50+ plans for my own API operations, and I just prefer having access to a wide range of API resources, across many different companies, with a variety of usage limitations and pricing based upon each individual resources. I really am getting tired of having to choose between bronze, gold, or platinum, and often getting priced out completely because I can scale to the next tier as a user.
I understand that companies like putting users into buckets, something that makes revenue predictable from month to month, or year to year, but as we consumer more APIs from many different providers, it would help reduce the complexity for us API consumers if you flattened the landscape. I really don't want to have to learn the difference between each of my provider's tiers. I just want access to THAT resource via an API, at a fair price--something that scales infinitely if at all possible (I want it all). Ultimately, I do not feel like API plans and tiers will scale to API economy levels. I think as API providers, we are still being pretty self-centered, and thinking about pricing as we see it, and we need to open up and think about how our API consumers will view us in a growing landscape of service providers--otherwise, someone else will.
As I pick up my earlier API pricing work, which has two distinct components: 1) all API resources and pricing available for a platform 2) the details of plans and tiers which a complex list of resources, features, and pricing fit into. It would be much easier to just track resources, the features they have, and the unit price available for each API. Then we could let volume, time-based agreements, and other aspects of the API contract help us quantify the business of APIs, without limiting things to just a handful of API contract plans and tiers, expanding the ways we can do business using APIs.
As an API provider, I get that a SaaS model has worked well to quantify predictable revenue in a way that makes sense to consumers, but after a decade or more, as we move into a more serverless, microservices, devops world, it seems like we should be thinking in a more modular way when it comes to the business of our APIs. I'm sure all you bean counters can get out of your comfort zone for a bit, and change up how you quantify access to your API resources, following the lead of API pioneers like Amazon, and just provide a master list of API, CLI, and console resources available for a competitive price.
I think way too much about the digital bits being transmitted online each day. I study the APIs that are increasingly being used to share these bits via websites, mobile, and other Internet-connected devices. These bits can be as simple as your messages and images or can be as complex as the inputs and outputs of algorithms used in self-driving cars. I think about bits at the level up from just the 1s and 0s, at the point where they start to become something more meaningful, and tangible--as they are sent and received via the Internet, using web technology.
The average person takes these digital bits for granted, and are not burdened with the technical, business, and political concerns surrounding each of these bits. For many other folks across a variety of sectors, these bits are valuable and they are looking to get access to as many of them as you can. These folks might work at technology startups, hedge funds, maybe in law enforcement or just tech-savvy hacker or activist on the Internet. If you hang out in these circles, data is often the new oil, and you are looking to get your hands on as much of it as you can, and are eager to mine it everywhere you possibly can.
In 2010, I started mapping out this layer of the web that was emerging, where bits were beginning to be sent and received via mobile devices, expanding the opportunity to make money from these increasingly valuable bits on the web. This move to mobile added a new dimension to each bit, making it even more valuable than they were before--it now possessed a latitude and longitude, telling us where it originated. Soon, this approach to sending and receiving digital bits spread to other Internet-connected devices beyond just our mobile phones, like our automobiles, home thermostats, and even wearables--to name just a few of the emerging areas.
The value of these bits will vary from situation to situation, with much of the value lying in the view of whoever is looking to acquire it. The value of a Facebook wall post is worth a different amount to an advertiser looking for a potential audience, then it will be to law enforcement looking for clues in an investigation, and let's not forget the value of this bit to the person who is posting it, or maybe their friends who are viewing it. When it comes to business in 2017, it is clear that our digital bits are valuable, even if much of this value is purely based on perception and very little tangible value in the real world. With many wild claims about the value and benefit of gathering, storing, and selling bits.
Markets are continually working to define the value of bits at a macro level, with many technology companies dominating the list, and APIs are defining the value of bits at the micro level--this is where I pay attention to things, at the individual API transaction level. I enjoy studying the value of individual bits, not because I want to make money off of them, but because I want to understand how those in power positions perceive the value of our bits and are buying and selling our bits at scale. Whether it is compute and storage in the cloud, or the television programs we stream, and pictures and videos we share in our homes, these bits are increasing in value, and I want to understand the process how everything we do is being reduced to a transaction.
When thinking about generating revenue generated from APIs it is easy to focus on directly charging for any digital resource being made available via the API. If it's an image, we charge per API call, and maybe the amount of MB transferred. If it's messaging, we charge per message. There are plenty of existing examples out there regarding how you directly charge for data, content, or algorithms using APIs, and an API way of doing business--look to Amazon, Twilio, and other pioneers.
Where there are fewer examples and less open discussions, is around the value of the operation level of APIs,Â and making these data available via APIs--yes APIs for APIs. Modern approaches to doing APIs are all about requiring each application to use an API key with each call they make, the logging of each request and response, possessing the identifying key for each application. This is how API providers are developing an awareness of who is accessing resources, how they are being put them to use, and specific details about each application, and maybe even the users involved.
Sometimes the value generated at this layer doesn't exist. Due to restrictive access models, and direct revenue models, there isn't much going on operationally, so there isn't much value generated. However, when there is heavy usage around APIs, the exhaust of the API management layer can become increasingly valuable. What are people searching for? What are applications most popular? Which geographic regions are the most active? There is a pretty lengthy laundry list of valuable data points being applied across modern API operations, that are helping API providers better understand what is going on, that aren't often being included as part of the API road map, and future revenue models.
Ok, let me pause here for a moment. I identify the value being generated at this layer because I see existing providers reaching this realization in their operations, as well as wanting to help other providers see the potential being achieved by successful API providers. I also acknowledge there is a lot of exploitation, abuse, and surveillance going on at this level, which is one of the biggest reasons I'm encouraging more transparency, observability, and discussion about this layer. I want API providers to understand the potential, but I also want API consumers and the end users of their applications to understanding what is going on at the API operational layer as well.Â
The current model I'm looking at through this lens currently is around my Open Referral Human Services Data Specification (HSDS) work, where I'm trying to help define the operational layer of human services APIs, as well as the direct API access to this critical public data. I am asking the question of how stewards of this very important data at the municipal level leverage APIs to make their valuable resources more accessible, and put to work where it is most needed, while also being able to generate and harness the valuable particles generated as part of an API exhaust system. What are people searching for? How are demographics evolving in a city, and how can city services shift and evolve too. Making the operational layer available via API so that it is available to key decision makers, even if those are private sector decisions makers who are required to pay for access to this intelligence--bringing critical new revenue streams for data stewards.
Let's pause again and be honest about the privacy concerns here. Access at this layer needs an additional level of scrutiny and care, over the direct access layers. Examples I'm concerned with can be seen in searches for Muslim religious services, or possibly LGBTQ services, and other information that could be used to violate the privacy and rights of application developers and end users. There are numerous privacy and security concerns at this level, but the inverse of these concerns also highlight the value of a data access exhaust system at this level. This is important information, that can provide some real time signals for both the public and private sector to consider more deeply.
I am purposely using the word exhaust here, specifically as a noun, as I don't think people are considering this layer, and may often see log files, and other ways data being generated in this as way as an unusable byproduct and output of web and mobile operations. I want to help people see the potential dangers of exhaust from API-centric data operations, but also understand that when it is captured, it can become valuable, similar to natural gas capture from recycling or landfill operations. There are some toxic aspects of API-driven data operations, but when measured and controlled, and made more observable, the dangerous aspects can be mitigated, and you might also find ways that other reuse and extraction that can also occur along the way.Â
I'm involved in some very interesting conversations with public data folks who are trying to push forward the conversation around sensible revenue generation by cities, counties, state, and the federal government using public data. I'm learning a lot from these conversations, resulting in the expansion and evolution my perceptions of how the API layer can help the government develop new revenue streams through making public data more accessible.Â
I have long been a proponent of using modern API management infrastructure to help government agencies generate revenue using public data. I would also add that I'm supportive of the crafting of sensible approaches to developing applications on top of public data and API in ways that generate a fair profit for private sector actors. I am also in favor of free and unfettered access to data, and observability into the platform operations, as well as ALL commercial interests developing applications on top of public data and APIs. I'm only in favor of this, when the right amount of observability is present--otherwise digital good olÂ boy networks form, and the public will lose.
API management is the oldest area of my API research, expanding into my other work to eventually defineÂ documentation, SDKs, communication, support, monetization, and API plans. This is where you define the business of API operations, organizing APIs into coherent catalogs, where you can then work to begin establishing a wider monetization strategy, as well as tiers and plans that govern access to data, content, and algorithms being made available via APIs. This is the layer of API operations I'm focusing on when helping government agencies better understand how they can get more in tune with their data resources, and identify potential partnerships and other applications that might establish new revenue streams.
A portion of thisÂ conversation that I am having was involved in the story from Anthony Williams about maybe government data shouldn't always be free, where the topic of taxation came up. One possible analogy for public data access and monetization was brought up as a reference to theÂ Vehicle-miles Traveled (VMT) tax, injecting the concept of taxation to my existing thoughts on revenue generation using API management. I've considered affiliate and reseller aspects to the API management layer before, applying percentage based revenue and payments on top of API access, but never thought about a government taxation layer existing here.
I thought my stance on revenue generation on public data using API management was controversial before, adding in concepts of taxation to the discussion is really going to invigorate folks who are in opposition to my argument. I'm sure there is a libertarian free web, open data advocate, smaller government Venn diagram in there somewhere. I'm not too concerned, as the monetization is already going on, I'm simply talking about making it more observable, and in favor of revenue generation for data stewards and government agencies. I'm confident that most won't folks in opposition won't even read this paragraph, as it's buried in the middle of this post. ;-)
I take no stance on which data, content, or algorithms should be taxed, or what that tax rate should be. I leave this to data stewards and policy makers. My objective is to just introduce folks to the concept, and marry with the existing approaches to using APIs to develop digital products and services in the private sector. However, if I was wearing my policy maker hat I would suggest thinking about this as a digital VAT tax, "that is collected incrementally, based on the surplus value, added to the price on the work at each stage of production, which is usually implemented as a destination-based tax, where the tax rate is based on the location of the customer."
My thoughts on a government tax at the API management layer are at an early stage. I am just exploring the concept on my blog--this is what I do as the API Evangelist. I'd love to hear your thoughts, on your blog. I am merely suggesting a digital VAT tax at the API contract layer around public data and APIs when commercial activities are involved. Eventually, I could see the concept spread to other sectors as the API economy becomes a reality, but I feel that public data provides us with a rich test bed for a concept like this. I'm considering reframing my argument about charging for commercial access to public data using APIs as taxing commercial usage of public data using APIs, allowing for tax revenue to fund future investment in public data and API efforts.
As I remove my API Evangelist hat and think about this concept, I'm not 100% sure if I'm in agreement with my argument. It will take a lot more polishing before I'm convinced that taxation should be included in the API management layer. I'll keep exploring, and play with a variety of potential use cases, and see if I can build a case for API taxation when public data is involved, and applications are generating surplus value in the API economy.Â
It made me happy to read the Rise of Non “VC compatible” SaaS Companies, and see that there are more sensible discussions going on around how to develop SaaS business, something that I hope spreads into the specifically API as a product startups and API service providers. I know that many of my readers think I'm anti-VC--I am not. Or may I'm anti-startup--I am not. I'm anti-VC and anti-startup ideology becoming the dominant religion, pushing out a lot of really good people and ideas who can't play that game.
When it comes to Silicon Valley, if you push back, you get excluded from the club, and there are waves of people who step up to tell you "not all startups are bad" or "not all VCs are bad"--I wish I could help you understand how this response makes you look. Of course, they aren't all bad, but there are bad ones, and there is a lot of rhetoric that this is the ONLY way you can build technology when it isn't. There are plenty of ways you can develop technology, and build a business without the VC approach, or the cult of the startup.
There are more instructions you should follow in the rise of the non-VC compatible SaaS companies story, but the author outlines four types of SaaS companies, which I think applies nicely to APIi companies, as many of them will be SaaS providers:
- Funded SaaS: companies which finance their business with VCs a.k.a equity against money. From early stage startups with no revenue to companies going public with hundreds of millions of dollars of ARR, the range is extremely wide.
- Bootstrapped “scaling” SaaS companies: SaaS companies which manage to pass the $10M ARR threshold without VC money. Ex: Mailchimp or Atlassian (which raise VC money but at a very late stage) have reached the hundreds of millions of dollars of ARR without VC money. These “unicorns among unicorns” are very rare.
- Bootstrapped SaaS companies: bootstrapped companies which manage to reach the $300k — $10M ARR range without VC money.
- Bootstrapped Micro SaaS: “1 to 3” people companies which operate in the $1k — $300k ARR range, without VC money.
There are some ideas that should go VC, but there are even more that should not. I want to see more talk like this about the funding realities of an API startups. A sensible discussion about what the goals are, how fast a company should grow, and whether the company is building a business to develop software solutions that we sell to customers, or a business to sell some large enterprise--hell, maybe even go IPO. These are all viable options for your business, I'm just looking for more discussion about the priorities. One more thing, you should be having this discussion out in the open with the customers you are supposedly selling your products and services to--this is the important part, not just the having of the discussion.
I'm not trying to get in the way of you getting super filthy rich. I'm just trying to strike a balance between you building a company, and the stability and availability of your APIs, and API tools and services, in an industry I depend on to make my living. You see, APIs have gotten a bad wrap for not being stable, and going away at any time. This isn't an API thing, this is a VC funded startup thing. When you are telling your companies that you are building a business, with products and services to purchase, and then everyone bakes your solutions into their solutions, and you go away--that sucks. If you tell people the truth from the beginning and are honest and communicative about what your plans are, people can build and plan accordingly--the problem isn't APIs going away, it is startup founders not caring.
I am just trying to strike a balance between your business aspirations, and those of the people I help convince to use your APIs. Many of them aren't trying to get rich. They are just trying to build a business, and get their work done with your service, tool, and API. Let's start having more honest and open conversation about the variety of business models available to use when building API startups, and remember that not everything is a VC-worthy idea, sometimes you are just a viable feature for regular business owners like me.
I am helping a customer think through their decision-making process around the adoption of a new API service, and while I am doing this I am spending the time to think through my own API adoption process. I like having checklists to consider when making new purchasing and integration decision. Sometimes I have an immediate need which is driven by emotion, and it can help to step back and think through a more rational checklist I established for myself on a previous date.
When I am approaching a new API that I think might move beyond just playing around, and actually have a viable business use case in my operations, I think through the following areas:
- Define Value - What value is being created by an API I'm looking to use.
- Define Needs - What needs do I have which using an API will solve.
- Define Options - What other solutions are there beyond this API.
- Think About Today - Is this an immediate need I have with days or weeks.
- Think About Tomorrow - Is this a long term need that will go on for years.
- Vet Company & People - More detail about the company, people, and investors.
- Define Partners - What does my the partnership landscape look like for the API.
- What Things Cost - What are things going to cost be for using an API.
- What You Can Afford - Can I really afford using this service, or not use.
- Develop Continuity Plan - What is the plan for ensuring stable operations using API.
- Develop Exit Plan - How will I severe a relationship and replace or deprecate need.
Sometimes I do not have everything in order before I pull the trigger. I may not completely though through what I will do when an API goes away, but I like at least having some notes written down about my frame of mind when I was integrating with any API. Having the checklist to think about at the time of integration and purchase, as well as my notes about evaluation around a vendor provides me with something I can revisit each month when paying the bill, or may as I encounter a new service, or possibly instability or lack of access to an API.
I am using this as part of my own API operations, but I'm also helping a client think through their API purchasing decisions, and hopefully, make the process a much healthier and educated one. I'm hoping this helps put much of the burden of API stability on the shoulders of us people who are actually making the decision, allowing us to be more mindful of who we integrate with, and set more informed and honest expectations around what APIs do, and where some of the current business models in place might help, or hurt our plans.
I closely watch the value the digital bits being exchanged via the Interwebz--it is what I do. @audreywatters always says that APIs are "reducing everything to a transaction", and I am interested understanding the value of these bits, what people are buying and selling them for, and how it keeps the Internet machine chugging along--for better or worse. As I watch Audrey battle with folks about the availability of content with her domain and experience my own shift in what should be made freely available by API providers, I'm left thinking about the damaging effects free has had on our world.
I feel like the seeds of this were set into motion by John Perry Barlow followers imparting their ideology on the web, but was something that was capitalized on during the Web 2.0 movement by tech giants like Google, Twitter, and Facebook when it came to leveling the playing field, giving them the competitive advantage they needed. It is very difficult to compete with FREE. Only certain companies can operate in this environment. It's a brilliant and cutthroat way of doing business, setting a tone for how business should be done in a way that keeps competitors out of the game. When the free and open Internet armies become wielded by this type of passive aggressive capitalism, the resulting zombie army becomes a pretty effect force for attacking any providers who are left operating in this oxygen-deprived environment.
These free zombie armies think the web should be free and openly accessible for them to do what they want, most notably build a startup, getting funding, and sell that startup to another company. Your detailed website of business listings, research into an industry, and other valuable exhaust that MUST remain free is ripe for picking, and inclusion into their businesses. The zombies rarely go picketing after tech giants, telling them that everything must remain free and available, they go after the small service provider who is trying to make a living and build an actual small business. If the tech giants sucking the oxygen out of space with FREE don't get you, the free and open zombies will pick you clean through a sustained, and systematic assault from Reddit and Hacker News.
I'm always amazed at the bipolar conversations I have with folks about how I manage to make a living doing my API research, how rich and detailed my work is, while also being asked to jump on a phone call to talk through my research, so it can be used in their startup, marketing, or infographic. Never being asked if they could pay me, and when I mention getting paid-- they often just scatter. This continuous assault on the web has pushed me to shift my views on what should be FREE, and what we publish and openly license on the web, as well as make available at the lowest tiers of our APIs. These are my valuable bits. I've managed to stay alive and make a living in a field where most analysts either burn out or are acquired and coopted. My bits are how I make a living, please stop demanding that they always be free. Not everyone can operate like Google, Facebook, or Twitter--sometimes things cost money to do well, and might not need to be done at scale.
This is a multipart story on monetizing public data using APIs. I have spent the last seven years studying over 75+ aspects of the API delivery lifecycle across companies, organizations, institutions, and government agencies. This project is designed to be a distillation of my work to help drive a conversation around sensible and pragmatic revenue generation using public data--allowing the city, county, state, and federal government agencies to think critically about how open data efforts can exist and grow. It lives as a standalone repository, as well as individual stories that are meant to stand on their own, while also contributing to an overall narrative about public data monetization.
While my primary income is not derived from developing software for sale, I have developed commercial software throughout my career, and actively maintain my own API driven technology platform for tracking on the API industry. This is my best attempt to put on my technology vendor hat on for a bit to better understand the concerns and perspective of the software vendors involved with the public data sector. There is a wide spectrum of technology vendors servicing the space, making this exercise difficult to generalize, but I wanted to take a shot at defending and jumpstarting the conversation at the commercial vendor level.
Commercial tech vendors are always at the frontline of discussion around monetization of public data, for better or worse. When open data activists push back on my work to understand how public data can be monetized, the most common response I have is that public data is already being monetized by commercial vendors, and my work is about shining a light on this, and not being in denial that it is already occurring everywhere. Here are some of my thoughts from the public data commercial vendor landscape:
- Investment In Data - As a technology company I am investing a significant amount of resources into our data, and the data of our customers. While views may greatly vary on how much ownership platform and technology operators have around the public data they touch, it can't be argued that commercial vendors play a significant role--the discussion should be more about how great of a role, and how much ownership is there.
- Investment in Software - Beyond the data, we are investing a significant amount of resources into software, that our customers use, and we use to manage our business internally. This is where we will keep most of the proprietary value generated around public data, although the door around the availability of open source tooling needs to remain open. Similar to data, the question is about how much ownership over software do I need as a commercial vendor and how much can I give back to the community.
- Lead Generation - I am interested in generating leads for new customers, and joining in new conversations that demonstrate the value of the products and services that my company brings to the table.
- Sell My Services - I am interested in selling my products and services, and my motivation is going to reflect this. No matter what our mission or marketing may say, I'm interested in generating a profit for my company, and its interests.
- Premium Services - Our domain expertise, and investment in data and software opens up the opportunity for us to offer premium services on top of public data operations. While our customers may not always pay directly for data storage and management, or even software licenses, the ability to sell premium services is valuable to everyone involved.
- Protect Intellectual Property - It is important to us that our intellectual property is protected in all conversations, and that the licensing of data and software is respected, and reflected in our partnerships. While perspectives on what is appropriate regarding intellectual property will vary, it is important that IP Is always an up-front part of the conversation.
- Investment in R&D - Commercial vendors are more likely to invest in research and development, helping push forward innovation around public data, something that isn't always possible unless there are clear revenue opportunities for commercial operators and clear communication and understanding with non-commercial stakeholders about what is possible, and being done.
- Consistent Support - One important thing commercial vendors bring to the table for government agencies, and non-commercial stakeholders are the opportunity for consistent support. As seasons change in government, commercial vendors can be there to fill gaps in business and technical support services, keeping important conversations moving forward.
I have to be 100% transparent here and stop to say that while I am advocating for revenue generation around public data, I'm not always a proponent of that revenue benefitting commercial interests. First, and foremost, I want revenue to benefit the public, secondarily the non-commercial stakeholders, then thirdly for the benefit commercial vendors. Making this argument from the commercial vendor perspective is possible for me, just not something I'm always going to be in full support of, and I will always insist on pushing back on aggressive behavior from commercial vendors to dominate the conversation, in favor of data stewards, and the public.
With that said, I'm a believer that commercial activity can benefit public data efforts. Sensible revenue can be generated from delivering services, products, and tooling developed around public data, while also investing back into data operators, owners, and stewards, and most importantly benefit those being served. Depending on where you operate in the public data space you will see this argument, or hopefully conversation, differently. This is just an attempt to look at things from the side of commercial vendors, and being honest and transparent about what the commercial interests are when it comes to public data.
You can keep an eye on my public data monetization research via a separate site--I will be adding each segment here on the blog, as well as the individual project website. You can participate via the Github repository I am using to manage all my work in this area.
If you think there is a link I should have listed here feel free to tweet it at me, or submit as a Github issue. Even though I do this full time, I'm still a one person show, and I miss quite a bit, and depend on my network to help me know what is going on.