4 days agoSnowflake or Databricks? BigQuery or Dataproc? Redshift or EMR?How to choose the product to build your data platform upon — An organization that seeks to get better at how they use data and AI will need to get their data architecture into shape. Assuming that you are going to be building your data platform on the cloud, you will find yourself trying to make the choice between Snowflake, Databricks, BigQuery…Data Warehouse9 min readData Warehouse9 min read
Published in Becoming Human: Artificial Intelligence Magazine·Nov 2Who should be doing machine learning at your company?The key question for infusing your organization with ML is “who” not “how”. — The big decision that you as a leader have to make on ML is who in your organization does different parts of the ML workflow. …Machine Learning4 min readMachine Learning4 min read
Published in Becoming Human: Artificial Intelligence Magazine·Sep 18No, You Don’t Need MLOpsKeep It Simple: the complexity of full MLOps is rarely needed — MLOps started from a straightforward problem statement — that the technical debt associated with ML models becomes intolerable if the models are not adjusted over time to account for changes in the environment. Since that 2015 observation, ML models and frameworks have been built that make it relatively easy to…Mlops12 min readMlops12 min read
Published in Towards Data Science·Aug 25How to Generate Images with Stable Diffusion in Seconds, for PenniesAnd the limitations (today!) of this approach — The authors of Stable Diffusion, a latent text-to-image diffusion model, have released the weights of the model and it runs quite easily and cheaply on standard GPUs. This article shows you how you can generate images for pennies (it costs about 65c to generate 30–50 images). Start a Vertex AI Notebook The Stable Diffusion model…Stable Diffusion6 min readStable Diffusion6 min read
Jul 30Thru-hiking the Enchantments, WashingtonA step-by-step guide to preparing for, and doing the hike. — I hiked the 18-mile one-way trail across the Enchantments in July 2022. In this article, I’ll talk about what the Enchantments are, how I prepared for the hike, the logistics of the hike, and share my experience of the hike itself. What are the Enchantments? The Enchantments is an area near Leavenworth in Central…Hiking16 min readHiking16 min read
Published in DataDrivenInvestor·Jul 17Why estimating the number of bots in Twitter is not an ML problem and 100 accounts per day is sufficientExplaining the fallacy in the lawsuit against Twitter — Some highly paid lawyers made this dubious argument in the filing against Twitter: Because there are really good bots, the lawsuit uses the words “false or spam” to refer to bad actors on Twitter. Still, “false or spam” is a mouthful, so I’ll use the word “bot” in this article. The argument …Twitter6 min readTwitter6 min read
Published in Level Up Coding·Jul 14How to Write a Pandas Dataframe as a PDFUse matplotlib to draw a Pandas Dataframe into one large PDF or a multi-page PDF — This is a problem I was faced with recently — I had a Pandas dataframe with carefully computed metrics and needed to “print” them to a PDF. While there is a to_markdown() and to_html() in Pandas, there is no easy way to create a PDF file. Goal What I wanted was…Pandas2 min readPandas2 min read
Published in Towards Data Science·Jul 6How to use Customer Lifetime Value (LTV) for data-driven transformationUse LTV to set goals for your business and devolve accountability downwards — If your business is undertaking a data-driven transformation, one of the first things you ought to address is what the purpose of that transformation is. Data-driven transformation should not be about short-term thinking such as squeezing out operational efficiencies. Instead, as suggested in the excellent book Converted by Neil Hoyne…Ltv12 min readLtv12 min read
Jul 4Books to read if you want to become a data scientistMoving from academia to industry — If you are a graduate student in a quantitative discipline (engineering, biology, even sociology), you can make it in industry as a data scientist. But you do need a bit of preparation because data science is more than just statistics. How to prepare for the interview You have the necessary mathematics and programming skills, but are…Data Science2 min readData Science2 min read
Published in DataDrivenInvestor·May 710x is about Thinking not about Grit or SkillsWhy a simple question is so effective at making you work smarter — You hear the term “10x” bandied about so much that its lost its purpose, which is to shock. It does not mean you need to work harder (grit or “not giving up”) or become more skilled (“10x engineers”). Instead, 10x is about working smarter, by changing your approach to a…10 X3 min read10 X3 min read
