Skip to main content

Donating rules_xcodeproj to the Mobile Native Foundation

Brentley Jones
Brentley Jones, Developer Evangelist @ BuildBuddy

Since the first commit of rules_xcodeproj, we’ve been committed to it being community driven and community owned. Early in rules_xcodeproj’s development, multiple companies started using the project, while also putting forth significant contributions. And all throughout its development we’ve solicited feedback from users on how the project is working for them, not working for them, and what we should be focusing on next.

Today we are taking the next step in this commitment; we are solidifying the community ownership of rules_xcodeproj by donating it to the Mobile Native Foundation!

Multiple Xcode Configurations with rules_xcodeproj 1.3

Brentley Jones
Brentley Jones, Developer Evangelist @ BuildBuddy

Today we released version 1.3.2 of rules_xcodeproj!

This is a pretty exciting release, as it adds support for multiple Xcode configurations (e.g. Debug and Release). Since early in rules_xcodeproj’s development, being able to have more than the default Debug configuration has been highly requested. We would have implemented support much sooner, but because rules_xcodeproj accounts for every file path and compiler/linker flag, in order to have rock solid indexing and debugging support, it wasn’t an easy task.

Introducing rules_xcodeproj 1.0

Brentley Jones
Brentley Jones, Developer Evangelist @ BuildBuddy

Almost exactly one year ago I wrote the first commit for rules_xcodeproj. Like a lot of software engineers, I’m pretty bad at estimating, and thought that I would be able to finish 1.0 in 2 to 4 months 😅. The longer development cycle was a result of an increased scope and level of quality that I came to expect for a proper 1.0 release. Over the course of the year, I believe the project has risen to meet my expectations, and today I’m happy to announce the release of version 1.0 of rules_xcodeproj!

Welcoming Son Luong Ngoc

Siggi Simonarson
Siggi Simonarson, Co-founder @ BuildBuddy

At BuildBuddy, our mission is to bring the world's best developer tools to every company. A big part of that mission involves building a team folks with deep industry experience helping companies build and scale their systems.

That's why we're so excited to share that Son Luong Ngoc has joined BuildBuddy as a Solutions Engineer. His work with Bazel at scale will directly benefit BuildBuddy users and customers.

Son Luong is a contributor to bazel, git, rules_go, rules_docker, bazel-gazelle, and more. He's also a regular contributor to the Bazel Slack and maintains an incredible blog that explains the inner workings of Bazel.

Son Luong joins us from Qarik, where he helped large enterprises with Bazel and scalable build infrastructure. Prior to Qarik, Son Luong was an SRE at Booking.com, where he helped scale their Bazel monorepo.

We look forward to working alongside Son Luong to build the future of developer tools.

Welcome to BuildBuddy, Son Luong!

Welcoming Jim Hollenbach

Siggi Simonarson
Siggi Simonarson, Co-founder @ BuildBuddy

To fulfill our mission of bringing the world's best developer tools to every company, the most imporant thing we can do is hire incredible Software Engineers with experience building and scaling great software. That's why we're excited to share that Jim Hollenbach is joining BuildBuddy's engineering team!

Jim joins us from Google, where he was a Staff Software Engineer. He spent over 10 years at Google working across Search, Maps, and more.

We look forward to working alongside Jim to build the future of developer tools.

Welcome to BuildBuddy, Jim!

Bazelcon 2022 Recap

Siggi Simonarson
Siggi Simonarson, Co-founder @ BuildBuddy

Earlier this month we wrapped up the 2022 Bazelcon conference in New York City. The conference featured talks by many prominent Bazel users including Lyft, Spotify, Tesla, Slack, Stripe, Tinder, Tecton, Coinbase, Uber, more.

Here's are some of the highlights from the conference:

  • We gave a talk previewing the upcoming BuildBuddy 3.0 release
  • Six companies that are BuildBuddy Enterprise customers presented on the main stage
  • We co-hosted a Bazelcon happy hour with Google Cloud that had over 200 attendees
  • We shared what we've been working on over the past year with rules_xcodeproj
  • We gave away 370 BuildBuddy shirts and 1600 BuildBuddy stickers
  • We finally got to meet so many of incredible customers & open source contributors in person!

Tweets, talks, and pictures from the event below!

How We Use ClickHouse to Analyze Trends Across Millions of Builds

Lulu Zhang
Lulu Zhang, Engineer @ BuildBuddy

When you use Buildbuddy with Bazel to build and test software, Buildbuddy captures information about each Bazel invocation, such as number of builds, build duration, remote cache performance, and more. Buildbuddy has a Trends page to visualize trends in this data over time.

The trends page allows you to see how improvements you are making to your builds affects your average build duration and other stats. It also exposes areas that might need improving. For example, if you see the cache hit rate go down over time, your build might have some non-deterministic build actions that could be improved, or some newly introduced dependencies that result in more frequent cache invalidations.

When we first created the Trends page, we used MySQL queries to aggregate build stats and generate the data we wanted to display. For a time this worked well, but we quickly ran into performance issues for customers that had very large numbers of builds. We were able to temporarily improve performance by adding various indices, and though this helped to reduce the number of rows read, it was not sufficient. Some customers do millions of builds monthly, and the Trends page (which can look back up to a year) for these customers was taking more than 20 minutes to load.

The queries behind the trends page require aggregation of multiple columns, such as cache hits and cache misses. A traditional row-based database like MySQL is not always ideal for such a use case. In row-based databases, data is stored row by row. When aggregating columns, more I/O seeks are required than a column-based database, which stores the data of each column in contiguous blocks. Moreover, column-based databases have a higher compression rate because consecutive values of the same column are of the same type and may repeat.

With a row-based store, we can see from this diagram that computing a sum of cache hit count would require us to load both block 1 and block 2. With a column-based store, all the cache hits data are stored in the same block.

Therefore, we felt that using ClickHouse, a column-based database, would improve the performance of required queries for the trends page. We validated ClickHouse’s performance against our use case: it took ClickHouse 0.317 seconds to process 1.5 million rows and calculate the stats. The same query took MySQL about 24 minutes.

One of our goals for data migration is to make sure the data is accurate. We added monitoring and compared data between MySQL and ClickHouse after we enabled double writing in production. One source of inconsistency was that data was inserted into ClickHouse both by the backfill script and production servers. Different to a traditional database, ClickHouse’s ReplacingMergeTree engine only deduplicates data in the background at an indeterminate time. As a result, we needed to manually run the OPTIMIZE operation to force ClickHouse to deduplicate data after the backfill was done. After we were confident in the data consistency, we finally enabled the Trends page to read from ClickHouse.

What's next

We are excited how ClickHouse unlocks more possibilities for us to provide analytical insights into builds, targets, tests and remote execution. For example, we want to add graphs that show how remote actions are spending most of their time. These insights can be used to guide remote execution performance optimizations.

We would love to hear your feedback about what stats and graphs you are interested in seeing. Join our Slack channel or email us at hello@buildbuddy.io with any questions, comments, or thoughts.

Welcoming Iain Macdonald

Siggi Simonarson
Siggi Simonarson, Co-founder @ BuildBuddy

To fulfill our mission of bringing the world's best developer tools to every company, we're intensely focused on hiring outstanding Software Engineers. That's why we're excited to share today that Iain Macdonald is joining BuildBuddy's engineering team!

Iain joins us from Google, where he spent over 10 years as an engineer working across the company from Gmail to Google Maps.

We look forward to working alongside Iain to build the future of developer tools.

Welcome to BuildBuddy, Iain!

Welcoming Maggie Lou

Siggi Simonarson
Siggi Simonarson, Co-founder @ BuildBuddy

To fulfill our mission of bringing the world's best developer tools to every company, we're expanding our team to keep up with our evergrowing customer base. That's why we're excited to share today that Maggie Lou is joining BuildBuddy's engineering team!

Maggie joins us from Thumbtack.

We look forward to working alongside Maggie to build the future of developer tools.

Welcome to BuildBuddy, Maggie!