Lesson: Data Driven Decisions with Mike Greenfield

Step #9 Data Infrastructure: In-house vs outsourcing

Investing in data infrastructure is something that I think is valuable when you are at a decent scale in terms of users. If you have 100 users, the most sophisticated A/B testing system in the world is not going to help you all that much, because you don't have enough data to do a lot with it. At the same time, if you have millions of users, there's a lot that you can do with A/B testing if you control everything, rather than outsourcing your data analysis and your testing to a third party.

The third party tools for doing A/B testing have certainly improved in the last four, five, six years. If I were building Circle of Moms today, I would at least consider farming pieces of that out, at least early on. At the same time, when you own it, when all that data is in your database and you can crunch it in whatever way you want, there's a big advantage there, and I think any company that has even five or six engineers should probably be building a lot of that stuff on their own, at least saving it to their database, so they can do whatever they want with it if they need to.

A/B testing can be a way of gauging interest relatively quickly. If you show a basic version of something that looks at interest and says, "Okay, how likely are people to click on this if you show this versus not showing this? Is that going to make them more engaged? Are they actually going to be interested in that?" that can help inform future product direction.