Skip to main content

15 posts tagged with "product"

View All Tags

· 3 min read
Lulu Zhang

When you use Buildbuddy with Bazel to build and test software, Buildbuddy captures information about each Bazel invocation, such as number of builds, build duration, remote cache performance, and more. Buildbuddy has a Trends page to visualize trends in this data over time.

The trends page allows you to see how improvements you are making to your builds affects your average build duration and other stats. It also exposes areas that might need improving. For example, if you see the cache hit rate go down over time, your build might have some non-deterministic build actions that could be improved, or some newly introduced dependencies that result in more frequent cache invalidations.

When we first created the Trends page, we used MySQL queries to aggregate build stats and generate the data we wanted to display. For a time this worked well, but we quickly ran into performance issues for customers that had very large numbers of builds. We were able to temporarily improve performance by adding various indices, and though this helped to reduce the number of rows read, it was not sufficient. Some customers do millions of builds monthly, and the Trends page (which can look back up to a year) for these customers was taking more than 20 minutes to load.

The queries behind the trends page require aggregation of multiple columns, such as cache hits and cache misses. A traditional row-based database like MySQL is not always ideal for such a use case. In row-based databases, data is stored row by row. When aggregating columns, more I/O seeks are required than a column-based database, which stores the data of each column in contiguous blocks. Moreover, column-based databases have a higher compression rate because consecutive values of the same column are of the same type and may repeat.

With a row-based store, we can see from this diagram that computing a sum of cache hit count would require us to load both block 1 and block 2. With a column-based store, all the cache hits data are stored in the same block.

Therefore, we felt that using ClickHouse, a column-based database, would improve the performance of required queries for the trends page. We validated ClickHouse’s performance against our use case: it took ClickHouse 0.317 seconds to process 1.5 million rows and calculate the stats. The same query took MySQL about 24 minutes.

One of our goals for data migration is to make sure the data is accurate. We added monitoring and compared data between MySQL and ClickHouse after we enabled double writing in production. One source of inconsistency was that data was inserted into ClickHouse both by the backfill script and production servers. Different to a traditional database, ClickHouse’s ReplacingMergeTree engine only deduplicates data in the background at an indeterminate time. As a result, we needed to manually run the OPTIMIZE operation to force ClickHouse to deduplicate data after the backfill was done. After we were confident in the data consistency, we finally enabled the Trends page to read from ClickHouse.

What's next

We are excited how ClickHouse unlocks more possibilities for us to provide analytical insights into builds, targets, tests and remote execution. For example, we want to add graphs that show how remote actions are spending most of their time. These insights can be used to guide remote execution performance optimizations.

We would love to hear your feedback about what stats and graphs you are interested in seeing. Join our Slack channel or email us at hello@buildbuddy.io with any questions, comments, or thoughts.

· 5 min read
Brandon Duffany

Using a remote cache is a great way to speed up your Bazel builds! But by default, Bazel uploads almost everything to the remote cache.

If your network is slow and your build artifacts are very large (like a docker image) this can lead to poor performance.

To address this, and make it easier to fix, we built the new cache requests card.

In this post we'll explore what insights this card can give you into your builds, as well as some fun details about how the card works under the hood.

· 4 min read
Siggi Simonarson

To celebrate Bazelcon 2021 (and our Bazelcon talk), we've got a lot in store for you in this release! We're excited to share that v2.7.0 of BuildBuddy is live on both Cloud Hosted BuildBuddy and open-source via Github, Docker, and our Helm Charts!

We've also updated the list of open source repositories that use BuildBuddy. If you have an open source repo that's using BuildBuddy, and would like to be included on the list - please fill out the quick form at the bottom of that page!

Thanks to everyone using open source, cloud-hosted, and enterprise BuildBuddy. We've made lots of improvements in this release based on your feedback.

· 5 min read
Brandon Duffany

In today's BuildBuddy v2.3 release, which is now live on BuildBuddy Cloud, we're launching BuildBuddy Workflows. BuildBuddy Workflows is a Continuous Integration (CI) solution for Bazel repositories hosted on GitHub (with support for other providers coming soon).

Like other CI solutions, Workflows give you the confidence that your code builds successfully and passes all tests before you merge pull requests or deploy a new release.

But because BuildBuddy Workflows were built for Bazel repos and tightly integrated with BuildBuddy RBE and Remote Caching, they are really fast.

· 5 min read
Siggi Simonarson

Our mission at BuildBuddy is to make developers more productive. When we released the first version of BuildBuddy a little over a year ago, we were blown away by the demand for tools and techniques for speeding up common developer workflows like building, testing, and debugging code. We've been working hard ever since - using our own tools to build the next generation of developer tooling for all.

Today we're excited to announce v2 of BuildBuddy! We've completely revamped our caching and remote build execution infrastructure to give our users and customers the one thing they care about above all else: speed.

· 3 min read
Siggi Simonarson

We're excited to share that v1.8.0 of BuildBuddy is live on Cloud Hosted BuildBuddy, Enterprise, and Open Source via GitHub, Docker, and our Helm Charts!

Thanks to everyone using open source, cloud-hosted, and enterprise BuildBuddy. We've made lots of improvements in this release based on your feedback.

A special thank you to our new open-source contributor:

  • Ashley Davies who contributed several pull requests to our Helm charts in order to make them easier to use in clusters that already have an Nginx controller deployed.

And a warm welcome to our three new team members!

  • Pari Parajuli who joins our engineering team as an intern who's currently studying at University of California, Berkeley.
  • Vadim Berezniker who joins our engineering team after 7 years at Google on the Google Cloud team.
  • Zoey Greer who joins us as a software engineer from the Google Search team.

We're excited to continue growing BuildBuddy and fulfill our mission of making developers more productive!

Our focus for this release was on reliability, performance, improved documentation, and making BuildBuddy easier to release and monitor.

· 3 min read
Siggi Simonarson

We're excited to share that v1.5.0 of BuildBuddy is live on both Cloud Hosted BuildBuddy and open-source via Github, Docker, and our Helm Charts!

Thanks to everyone using open source, cloud-hosted, and enterprise BuildBuddy. We've made lots of improvements in this release based on your feedback.

A special thank you to our new open-source contributor:

  • Corbin McNeely-Smith who contributed to making our auth flow more resilient to error cases, and made our health-check handlers more flexible to support different load-balancers.

Our focus for this release was on giving users more visibility into test flakiness, monitoring & scaling improvements, and security hardening.

· 4 min read
Siggi Simonarson

We're excited to share that v1.4.0 of BuildBuddy is live on both Cloud Hosted BuildBuddy and open-source via Github and Docker!

Thanks to everyone using open source, cloud-hosted, and enterprise BuildBuddy. We've made lots of improvements in this release based on your feedback.

A special thank you to our new contributors who we'll soon be sending BuildBuddy t-shirts and holographic BuildBuddy stickers:

  • Daniel Purkhús who enabled environment variable expansion in BuildBuddy config files & more

  • Joshua Katz who added support for auto-populating build metadata from GitLab CI invocations

Our focus for this release was on giving users new tools to share, compare, analyze, and manage BuildBuddy invocations - as well as major performance and reliability improvements to our remote build execution service.

We're also excited to share that over the coming weeks and months, we'll be open sourcing much more of BuildBuddy - including our remote build execution platform. At BuildBuddy we're firmly committed to open source and believe that a transparent and open model is the only way to build truly great developer infrastructure for all.

· 4 min read
Siggi Simonarson

We're excited to share that v1.3.0 of BuildBuddy is live on both Cloud Hosted BuildBuddy and open-source via Github and Docker!

Thanks to everyone using open source and cloud-hosted BuildBuddy. We’ve made lots of improvements in this release based on your feedback.

Our focus for this release was on giving users new tools to improve build performance, debug cache hits, and a completely redesigned Cloud & Enterprise experience.

· 5 min read
Siggi Simonarson

Excited to share that v1.2.1 of BuildBuddy is live on both Cloud Hosted BuildBuddy and open source via Github and Docker!

Thanks to everyone that has tested open source and cloud-hosted BuildBuddy. We've made lots of improvements in this release based on your feedback.

A special welcome to our newest contributor and team member:

  • Brandon Duffany - Brandon is an ex-Googler and Cornell alumn who's worked as a software engineer on Google Assistant and Google Ads. He'll start off focused on surfacing better profiling and timing information to help users track down and fix slow builds!

Our focus for this release was on expanding access to BuildBuddy as well as improving scalability and performance.

We're also excited to announce that we're expanding the BuildBuddy Cloud free tier. BuildBuddy Cloud is now free for teams of up to 3 engineers in addition to being free for individuals open source projects of any size.

· 4 min read
Siggi Simonarson

Excited to share that v1.1.0 of BuildBuddy is live on both Cloud Hosted BuildBuddy and open source via Github and Docker!

Thanks to everyone that has tested open source and cloud-hosted BuildBuddy. We've made lots of improvements in this release based on your feedback.

A special thank you to our new contributors:

Our focus for this release was on our new Remote Build Execution platform. This release marks a huge step in fulfilling our mission of making developers more productive by supporting the Bazel ecosystem.

BuildBuddy's Remote Build Execution platform supports executing your Bazel build and tests in parallel across thousands of machines with automatic scaling, support for custom Docker images, and more. We've been iterating on and testing BuildBuddy RBE for months with companies of different sizes, and are excited to now make it available to everyone.

· 3 min read
Siggi Simonarson

Excited to share that v1.0.6 of BuildBuddy is live on both Cloud Hosted BuildBuddy and open source via Github and Docker!

Thanks to all of you that have been using open source and cloud-hosted BuildBuddy. We've made lots of improvements in this release based on your feedback.

A special thank you to our new contributors:

Our three major focuses for this release were on a better test results view, certificate based authentication, and our new results-store API.

We also laid a lot of groundwork for remote build execution in this release, which will be available in the coming weeks.

· 4 min read
Siggi Simonarson

We released our initial open source version of BuildBuddy to the Bazel community last month and have received a ton of interest, support, and feedback. We really appreciate everyone who's taken the time to kick the tires and try it out!

We're excited to share that BuildBuddy has been run on-prem at over 20 companies, and hundreds more developers have tried our cloud-hosted version.

People have found the shareable invocation links particularly useful when debugging builds remotely with co-workers while working from home. No more pasting console outputs into Pastebin!

We've taken all of the feedback we've gotten and made lots of improvements to both the open source and enterprise versions of BuildBuddy.

Our three major focuses for this release were on better build artifact handling, better test support, and enterprise authentication. We hope these changes help you continue to build and debug software faster. Keep the feedback coming!

· 2 min read
Siggi Simonarson

BuildBuddy is a managed Bazel build system --- it brings a Google-style development environment to any company in minutes.

The first product we've built is an open-core Bazel build log viewer. It consumes the Bazel build event protocol and makes logs accessible through a user-friendly web UI. If you're familiar with Google's Sponge tool - it's basically an open source version of that. It's available in 3 forms:

  • Open source - You can find the core product open source and free to use with an MIT license on GitHub.
  • Cloud hosted - We also offer a cloud hosted version of the product at buildbuddy.io. Plans range from a free version for personal use to an enterprise version suited for companies with complex needs.
  • On-prem - We offer an enterprise version of the product that you can run on-premises or in your own cloud. This version comes with all the bells and whistles - like dashboards, user accounts, and dedicated support.

The Bazel build log viewer is just the start. We're working on a shared build cache, remote build execution, and more.

Our end goal is to take the pain out of managing Bazel, so you can focus on building your product.

Reach out to us at hello@buildbuddy.io if you're interested, we'd love to chat!