Published in HashmapInc·PinnedBuilding ML PipelinesKubernetes with Argo for the Win — Kubernetes is a generally untapped compute resource. It is that unfamiliar territory between IaaS and PaaS. Many engineers are apprehensive of using it, IT management is fearful of adding it to the stack, and technologists keep begging for it to be used — but why? Another abstraction for pipeline code…Mlops10 min readMlops10 min read
Published in Contrarian Software Engineering·5 days agoThe Guide to Writing Good SoftwareTechnical debt sucks — So… I will be presenting here and within this series my views and opinions and reflect a great degree on how I address the bigger picture of tech debt. Nothing here is negative, it is all positive — we all want to grow! But don’t always know how. …Software Engineering2 min readSoftware Engineering2 min read
Published in HashmapInc·Jan 8, 2021Introducing Hashmap Data MigratorOpen Source Tooling to Move Your Data Warehouse to the Cloud — by John Aven and Jhimli Bora Choosing the right tools can be difficult when planning to move data to the cloud or any migration of a large data asset or assets. Additionally, the tools chosen to transport data in your new system may not support your current data systems —…Data Engineering9 min readData Engineering9 min read
Published in HashmapInc·Dec 4, 2020Exploring DevOps OptionsWorking with GitLab — Here, at Hashmap, we use GitLab as one of our many version control platforms. Yes, we employ more than one — we are a consulting firm — and as a consulting firm, it pays to be familiar with a plethora of the tools available out there. However, the core of…Gitlab Ci5 min readGitlab Ci5 min read
Published in HashmapInc·Nov 19, 2020Building ML PipelinesDockerizing Your Code — Machine learning continues to evolve. While many folks are moving towards more modern approaches, there is still an apprehension around using Docker and Docker related technologies in much of data science. …Docker10 min readDocker10 min read
Published in HashmapInc·Nov 19, 2020Building ML PipelinesWhat is a DAG? — In modern computing solutions, the concept of a DAG or Directed Acyclic Graph is central. While the term DAG has become quite the buzz word: understanding what they are, how they are used in computing, and how/where they show up in data science and machine learning is not just buzz…Airflow6 min readAirflow6 min read
Published in HashmapInc·Nov 5, 2020Why I Love MLflowAnd You Should Too — Whether you are a data scientist, an ML engineer, a data analytics manager, a compliance officer, or the CEO (or anyone else whose job depends or will eventually) — you need to know what MLflow is and why you should be using it. When a data scientist builds a model…Mlops6 min readMlops6 min read
Published in HashmapInc·Jan 31, 2020Data Science Repo Scaffolding for the Next Generation#3 in the Evolving Data Science Series — Welcome back to the Evolving Data Science series. If you haven’t already read the previous entries, please find them below and read them at your leisure. There is no ‘ordering’ in this series. …Machine Learning3 min readMachine Learning3 min read
Published in HashmapInc·Jan 30, 2020Snowflake Solution Anti-Patterns: Spark is My HammerHadoop is Not a Cloud Data Warehouse — Spark has been the answer to processing all big data in memory but is that still true? In the world of big data, Spark was the data engineer’s answer to solving big data needs that Hadoop demanded. Loading large amounts of data into memory and performing SQL and dataflow type…Cloud Computing3 min readCloud Computing3 min read
Published in HashmapInc·Jan 29, 2020Snowflake Solution Anti-Patterns: The Probable Data ScientistThat Wasteful ‘Parquet’ Layer — Those of us working within the big data space have been trained to build out various layers within a ‘data lake’. These layers, usually three to five, typically consist of the following (common alternate naming conventions included): Raw/Landing/Stage Standardized/Normalized Cleansed/Prepared Modeled/Provisioned Published/Reporting/Consumption/BI Occasionally, some of these layers get combined depending…Snowflake3 min readSnowflake3 min read
