{"id":820,"date":"2024-02-06T14:00:02","date_gmt":"2024-02-06T14:00:02","guid":{"rendered":"https:\/\/fde.cat\/index.php\/2024\/02\/06\/dotslash-simplified-executable-deployment\/"},"modified":"2024-02-06T14:00:02","modified_gmt":"2024-02-06T14:00:02","slug":"dotslash-simplified-executable-deployment","status":"publish","type":"post","link":"https:\/\/fde.cat\/index.php\/2024\/02\/06\/dotslash-simplified-executable-deployment\/","title":{"rendered":"DotSlash: Simplified executable deployment"},"content":{"rendered":"

We\u2019ve open sourced <\/span>DotSlash<\/span><\/a>, a tool that makes large executables available in source control with a negligible impact on repository size, thus avoiding I\/O-heavy clone operations.<\/span>
\nWith DotSlash, a set of platform-specific executables is replaced with a single script containing descriptors for the supported platforms. DotSlash handles transparently fetching, decompressing, and verifying the appropriate remote artifact for the current operating system and CPU.<\/span>
\nAt Meta, the overwhelming majority of DotSlash files are generated and committed to source control via automation, so we are also releasing a complementary GitHub Action to assemble a comparable setup outside of Meta.<\/span>
\nDotSlash is written in Rust for performance and is cross-platform.<\/span><\/p>\n

At Meta, we have a vast array of first-party and third-party command line tools that need to be available across a diverse range of developer environments. Reliably getting the appropriate version of each tool to the right place can be a challenging task.<\/span><\/p>\n

For example, the source code for many of our first-party tools lives alongside the projects that leverage them inside our <\/span>massive monorepo<\/span><\/a>. For such tools, the standard practice is to use <\/span>buck2 run<\/span><\/a> to build and run executables from source, as necessary. This has the advantage that tools and the projects that use them can be updated atomically in a single commit.<\/span><\/p>\n

While we use extensive caching and <\/span>remote execution<\/span><\/a> to provide our developers with fast builds, there will always be cases where buck2 run\u00a0is going to be considerably slower than running the prebuilt binary directly. While we leverage a <\/span>virtual filesystem<\/span><\/a> that reduces the drawbacks of checking large binaries into source control compared to a traditional physical filesystem, there are still pathological cases that are best avoided by keeping such files out of the repository in the first place. (This practice also eliminates a large class of code provenance issues.)<\/span><\/p>\n

Further, not everything we use is built from source, nor do all of our tools live in source control. For example, there is the case of buck2 <\/span>itself, which needs to be pre-built for developers and readily available on the $PATH<\/span>\u00a0for convenience. For core developer tools like <\/span>Buck2<\/span><\/a> and <\/span>Sapling<\/span><\/a>, we use a <\/span>Chef recipe<\/span><\/a> to deploy new versions, installing them in \/usr\/local\/bin (or somewhere within the appropriate %PATH$% <\/span>on Windows) across a variety of developer environments.<\/span><\/p>\n

While this approach is reasonable for commonly-used executables, it is not a great fit for the long tail of tools. That is, while it might be convenient to install everything a developer might need in \/usr\/local\/bin<\/span>\u00a0by default, this could easily add up to tens or hundreds of gigabytes of disk, very little of which will end up being executed, in practice. In turn, this makes Chef runs more expensive and prone to failure.<\/span><\/p>\n

Introducing DotSlash<\/span><\/h2>\n

DotSlash attempts to solve many of the problems described in the previous section. While <\/span>we do not claim it is a silver bullet<\/span><\/a>, we have found it to be the right solution for many of our internal use cases. At Meta, DotSlash is executed <\/span>hundreds of millions of times per day<\/span> to deliver a mix of first-party and third-party tools to end-user developers as well as hermetic build environments.<\/span><\/p>\n

The idea is fairly simple: we replace the contents of a set of platform-specific, heavyweight executables with a single lightweight text file that can be read by the dotslash <\/span>command line tool (which must be installed on the user\u2019s $PATH<\/span>). We call such a file a <\/span>DotSlash file. It contains the information DotSlash needs to fetch and run the executable it replaces for the host platform. By convention, a DotSlash file maintains the name of the original file rather than calling attention to itself via a custom file extension. Instead, it aspires to be a transparent wrapper for the original executable. To that end, a DotSlash file is <\/span>required<\/span> to start with #!\/usr\/bin\/env dotslash<\/span>\u00a0(even on Windows) to help maintain this illusion.<\/span><\/p>\n

The following is a hypothetical DotSlash file named node <\/span>that is designed to run v18.19.0 of Node.js. Note that users across x86 Linux, x86 macOS, and ARM macOS can all run the <\/span>same<\/span> DotSlash file, as DotSlash will take care of doing the work to select the appropriate executable for the host on which it is being run. In this way, DotSlash simplifies the work of cross-platform releases:\u00a0<\/span><\/p>\n

\n<\/p>\n

In this example, the workflow DotSlash runs through when executing node<\/span>\u00a0looks like:\u00a0<\/span><\/p>\n

See the <\/span>How DotSlash Works<\/span><\/a> documentation for details.<\/span><\/p>\n

Because of how <\/span>#!<\/span> works on Mac and Linux, when a user runs .\/node –version, <\/span>the invocation effectively becomes dotslash .\/node –version<\/span>. DotSlash requires that its first argument is a file that starts with #!\/usr\/bin\/env dotslash<\/span>, as mentioned above. Once it verifies the header, it uses a <\/span>lenient JSON parser<\/span><\/a> to read the rest of the file. DotSlash finds the entry in the “platforms”<\/span>\u00a0<\/span>section that corresponds to the host it is running on.<\/span><\/p>\n

DotSlash uses the information in this entry and hashes it to compute a corresponding file path (that doubles as a key) in the user\u2019s local DotSlash cache. DotSlash attempts to exec <\/span>the corresponding file, replacing argv0 <\/span>with the path to the DotSlash file and forwarding the remaining command line arguments (–version<\/span>, in this example) to the exec <\/span>invocation.<\/span><\/p>\n

If the target executable is in the cache, the user immediately runs Node.js as originally intended. In the event of a cache miss (indicated by exec <\/span>failing with ENOENT<\/span>), DotSlash uses the information from the DotSlash file to determine the URL it should use to fetch the artifact containing the executable as well as the size and digest information it should use to verify the contents. If this succeeds, the verified artifact is atomically mv\u2018d <\/span>into the appropriate location in the DotSlash cache and the exec <\/span>invocation is performed again. Note that DotSlash uses <\/span>advisory file locking<\/span><\/a> to avoid making duplicate requests even if DotSlash files requiring the same artifact are run concurrently.<\/span><\/p>\n

Note that it is common to have multiple DotSlash files refer to the same artifact, <\/span>such as a <\/span>.tar.zst<\/span> file<\/span><\/a>, while each DotSlash file maps to a distinct entry within the archive. For example, suppose <\/span>node-v18.19.0-darwin-arm64.tar.gz<\/span> is a compressed <\/span>tar<\/span> file that contains many entries, including node\u00a0, npm\u00a0, and npx<\/span>. The DotSlash file for <\/span>node<\/span> would be as follows:<\/span><\/p>\n

#!\/usr\/bin\/env dotslash<\/p>\n

{
\n “name”: “node-v18.19.0”,
\n “platforms”: {
\n “macos-aarch64”: {
\n “size”: 40660307,
\n “hash”: “blake3”,
\n “digest”: “6e2ca33951e586e7670016dd9e503d028454bf9249d5ff556347c3d98c347c34”,
\n \/\/ Note the difference from the previous example where “format”: “zst” has been
\n \/\/ replaced with “format”: “tar.gz”, which specifies what type of decompression
\n \/\/ logic to use as well as the path within the decompressed archive to run when
\n \/\/ this DotSlash file is executed.
\n “format”: “tar.gz”,
\n \/\/ Assuming node-v18.19.0-darwin-arm64.tar.gz contains node, npm, and npx in the
\n \/\/ node-v18.19.0-darwin-arm64\/bin\/ folder within the the archive, the following
\n \/\/ is the only line that has to change in the DotSlash file that represents
\n \/\/ those other executables.
\n “path”: “node-v18.19.0-darwin-arm64\/bin\/node”,
\n “providers”: [
\n {
\n “url”: “https:\/\/nodejs.org\/dist\/v18.19.0\/node-v18.19.0-darwin-arm64.tar.gz”
\n }
\n ]
\n },
\n \/* other platforms omitted for brevity *\/
\n }
\n}<\/p>\n

As noted in the comments, the only change in the DotSlash files for npm\u00a0<\/span>and npx <\/span>would be the “path”<\/span>\u00a0entry. Because the artifact for all three DotSlash files would be the same, whichever DotSlash file was run first would fetch the artifact and put it in the cache whereas all subsequent runs of <\/span>any<\/span> of the three DotSlash files would leverage the cached entry.<\/span><\/p>\n

This technique is often used to ensure that a set of complementary executables is released together. Further, because the archive will be decompressed in its own directory, it may also contain resource files (or library files, such as .dll <\/span>files that need to live alongside .exe <\/span>files on Windows) that will be unpacked using the directory structure specified by the archive. This also makes DotSlash a good fit for distributing executables that are not binaries, but trees of script files, which is common for Node.js or Python.<\/span><\/p>\n

Generating DotSlash files<\/span><\/h2>\n

At Meta, most DotSlash files are produced as part of an automated build pipeline. Our continuous integration (CI) system supports special configuration for DotSlash jobs where a user must specify:<\/span><\/p>\n

A set of builds to run (these can span multiple platforms).<\/span>
\nThe resulting generated artifacts to publish to an internal blobstore.<\/span>
\nThe DotSlash files in source control to update with entries for the new artifacts.<\/span>
\nThe conditions under which the job should be triggered (this is analogous to <\/span>workflow triggers on GitHub<\/span><\/a>).<\/span><\/p>\n

The result of such a job is a proposed change to the codebase containing the updated DotSlash files. At Meta, we call such a change a <\/span>\u201cdiff,\u201d<\/span><\/a> though on GitHub, this is known as a <\/span>pull request<\/span><\/a>. Just like an ordinary human-authored diff at Meta, putting it up for review triggers a number of jobs that include linters, automated tests, and other tools that provide signal on the proposed change. For a DotSlash diff, if all of the signals come back clean, the diff is automatically committed to the codebase without further human intervention.<\/span><\/p>\n

See the Generating DotSlash Files at Meta<\/a> documentation for details.<\/p>\n

The script we use to generate DotSlash files injects metadata about the build job that makes it straightforward to trace the provenance of the underlying artifacts. The following is a hypothetical example of a generated DotSlash file for the <\/span>CodeCompose<\/span><\/a> LSP built from source at a specific commit in clang-opt <\/span>mode. Note the “metadata” <\/span>entries in the DotSlash file will be ignored by the dotslash<\/span>\u00a0CLI, but we include them as structured data so they can be parsed by other tools to facilitate programmatic audits:<\/span><\/p>\n

#!\/usr\/bin\/env dotslash<\/p>\n

\/\/ @generated SignedSource<<d8621e8ccbd7a595a3018e6a070be9c0>>
\n\/\/ https:\/\/yarnpkg.com\/package?name=signedsource can be used to
\n\/\/ generate and verify the above signature to flag tampering
\n\/\/ in generated code.<\/p>\n

{
\n “name”: “code-compose-lsp”,
\n \/\/ Added by automation.
\n “metadata”: {
\n “build-info”: {
\n “job-repo”: “fbsource”,
\n “job-src”: “dotslash\/code-compose-lsp.star”,
\n \/\/ It is considered best practice to build the artifacts for
\n \/\/ all platforms from the same commit within a DotSlash file.
\n “commit”: {
\n “repo”: “fbsource”,
\n “scm”: “sapling”,
\n “hash”: “0f9e3d9e189bf393f7f9d0b6879361cd76fcdcd0”,
\n “date”: “2024-01-03 20:07:54 PST”,
\n “timestamp”: 1704341274
\n }
\n }
\n },
\n “platforms”: {
\n “linux-x86_64”: {
\n “size”: 2740736,
\n “hash”: “blake3”,
\n “digest”: “fc8a3ade56a97a6e73469ade1575e8f8e33fda99fbf6df429d555e480d6453d0”,
\n “format”: “zst”,
\n “providers”: [
\n {
\n “type”: “meta-cas”,
\n “key”: “fc8a3ade56a97a6e73469ade1575e8f8e33fda99fbf6df429d555e480d6453d0:2740736”
\n }
\n ]
\n \/\/ Added by automation.
\n “metadata”: {
\n “build-command”: [
\n “buck2”,
\n “build”,
\n “–config-file”,
\n “\/\/buildconfig\/clang-opt”,
\n “\/\/codecompose\/lsp\/cli:code-compose-lsp”
\n ]
\n }
\n },
\n \/\/ additional platforms…
\n }
\n}<\/p>\n

Without DotSlash, a developer would have to run buck2 build –config-file \/\/buildconfig\/clang-opt \/\/codecompose\/lsp\/cli:code-compose-lsp<\/span>\u00a0to build and run the LSP from source, which could be a slow operation depending on the size of the build, the state of the build cache, etc. With DotSlash, the developer can run the optimized LSP as quickly as they can fetch and decompress it from the specified URL, which is likely much faster than doing a build.<\/span><\/p>\n

Another thing you may have noticed about this example is that the “key”<\/span>\u00a0is not an ordinary URL, but an identifier that happens to be the concatenation of the BLAKE3 hash and the size of the specified artifact. This is because “type”: “meta-cas” <\/span>indicates that this artifact must be fetched via a <\/span>custom provider<\/span> in DotSlash, which is specialized fetching logic built into DotSlash that has its own identifier scheme. In this case, the artifact would be fetched from Meta\u2019s in-house content-addressable storage (CAS) system, which uses the artifact hash+size as a key.<\/span><\/p>\n

While we do not provide the code for the meta-cas<\/span> provider in the open source version of DotSlash, we do include one custom provider out-of-the-box beyond the default http <\/span>provider.<\/span><\/p>\n

Using DotSlash with GitHub releases<\/span><\/h2>\n

While DotSlash is generally useful for fetching an executable from an arbitrary URL and running it, we have found the combination of DotSlash and CI to be particularly powerful. To that end, we include custom tooling to facilitate generating DotSlash files for GitHub releases. To ensure DotSlash can fetch artifacts from private GitHub repositories as well as GitHub Enterprise instances, DotSlash includes a custom provider for GitHub releases that includes an appropriate authentication token when fetching artifacts.<\/span><\/p>\n

For example, suppose you have existing workflows for building your release artifacts and publish them via gh release upload<\/span>. For simplicity, let\u2019s assume these are named linux-release<\/span>, macos-release<\/span>, and windows-release<\/span>. To create a single DotSlash file that includes the artifacts from all three platforms you would introduce a new <\/span>GitHub Action<\/span><\/a> that leverages the workflow_run<\/span>\u00a0trigger so it fires whenever one of these release workflows succeeds. (Note that <\/span>GitHub\u2019s documentation states<\/span><\/a>: \u201cYou can\u2019t use workflow_run<\/span>\u00a0to chain together more than three levels of workflows,\u201d so check the depth of your workflow graph if your workflow is not firing.)<\/span><\/p>\n

The .yml<\/span>\u00a0file to define the new workflow would look like this:<\/span><\/p>\n

name: Generate DotSlash File<\/p>\n

on:
\n workflow_run:
\n # These must match the names of the workflows that publish
\n # artifacts to your GitHub Release.
\n workflows: [linux-release, macos-release, windows-release]
\n types:
\n – completed<\/p>\n

jobs:
\n create-dotslash-file:
\n name: Generating DotSlash File
\n runs-on: ubuntu-latest
\n if: ${{ github.event.workflow_run.conclusion == ‘success’ }}
\n steps:
\n – uses: facebook\/dotslash-publish-release@v1
\n env:
\n # This is necessary because the action uses
\n # `gh release upload` to publish the generated DotSlash file(s)
\n # as part of the release.
\n GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
\n with:
\n # Additional file that lives in your repo that defines
\n # how your DotSlash file(s) should be generated.
\n config: .github\/workflows\/dotslash-config.json
\n # Tag for the release to to target.
\n tag: ${{ github.event.workflow_run.head_branch }}<\/p>\n

Because <\/span>inputs to GitHub Actions<\/span><\/a> are limited to string values, facebook\/dotslash-publish-release<\/span> takes config<\/span>, which is a path to a JSON file in the repo that supports a rich set of configuration options for generating the DotSlash files. The other required input is the ID of the release, which in GitHub, <\/span>is defined by a Git tag<\/span><\/a>. When the action is run, it will check to see whether all of the artifacts specified in the config are present in the release, and if so, will generate the appropriate DotSlash files and add them to the release.<\/span><\/p>\n

For example, consider an open source project like <\/span>Hermes<\/span><\/a> where a <\/span>release<\/span><\/a> includes a number of platform-specific .tar.gz <\/span>files, each containing a handful of executables (hermes<\/span>, hdb<\/span>, etc.). To create a separate an individual DotSlash file for each executable, the JSON configuration for the action would be:<\/span><\/p>\n

{
\n “outputs”: {<\/p>\n

“hermes”: {
\n “platforms”: {
\n “macos-x86_64”: {
\n “regex”: “^hermes-cli-darwin-“,
\n “path”: “hermes”
\n },
\n “macos-aarch64”: {
\n “regex”: “^hermes-cli-darwin-“,
\n “path”: “hermes”
\n },
\n “linux-x86_64”: {
\n “regex”: “^hermes-cli-linux-“,
\n “path”: “hermes”
\n },
\n “windows-x86_64”: {
\n “regex”: “^hermes-cli-windows-“,
\n “path”: “hermes.exe”
\n }
\n }
\n },<\/p>\n

“hdb”: {
\n “platforms”: {
\n “macos-x86_64”: {
\n “regex”: “^hermes-cli-darwin-“,
\n “path”: “hdb”
\n },
\n “macos-aarch64”: {
\n “regex”: “^hermes-cli-darwin-“,
\n “path”: “hdb”
\n },
\n “linux-x86_64”: {
\n “regex”: “^hermes-cli-linux-“,
\n “path”: “hdb”
\n },
\n “windows-x86_64”: {
\n “regex”: “^hermes-cli-windows-“,
\n “path”: “hdb.exe”
\n }
\n }
\n },<\/p>\n

\/\/ Additional entries for hvm, hbcdump, and hermesc…<\/p>\n

}
\n}’<\/p>\n

Each entry in “outputs”<\/span> corresponds to the name of a DotSlash file that will be added to the release. The “platforms”<\/span> for each entry defines the “platforms”<\/span>\u00a0that should be present in the generated DotSlash file. The action uses the “regex”<\/span> to identify the file in the GitHub release that should be used as the backing artifact for the entry. Assuming the artifact is an \u201carchive\u201d of some sort (.tar.gz<\/span>, .tar.zst<\/span>, etc.), the “path”<\/span>\u00a0indicates the path within the archive that the DotSlash file should run.<\/span><\/p>\n

In this particular case, Hermes does not provide an ARM-specific binary for macOS, so the “macos-aarch64”<\/span> entry is the same as the “macos-x86_64”<\/span>one. Though if that changes in the future, a simple update to “regex”<\/span>\u00a0to distinguish the two binaries is all that is needed.<\/span><\/p>\n

Note that the action will take responsibility for computing the digest for each binary. In this example, the resulting DotSlash file for hermes <\/span>would be:<\/span><\/p>\n

#!\/usr\/bin\/env dotslash<\/p>\n

{
\n “name”: “hermes”,
\n “platforms”: {
\n “linux-x86_64”: {
\n “size”: 47099598,
\n “hash”: “blake3”,
\n “digest”: “8d2c1bcefc2ce6e278167495810c2437e8050780ebb4da567811f1d754ad198c”,
\n “format”: “tar.gz”,
\n “path”: “hermes”,
\n “providers”: [
\n {
\n “url”: “https:\/\/github.com\/facebook\/hermes\/releases\/download\/v0.12.0\/hermes-cli-linux-v0.12.0.tar.gz”
\n },
\n {
\n “type”: “github-release”,
\n “repo”: “facebook\/hermes”,
\n “tag”: “v0.12.0”,
\n “name”: “hermes-cli-linux-v0.12.0.tar.gz”
\n }
\n ],
\n },
\n \/\/ additional platforms…
\n }
\n}<\/p>\n

Note that there are two entries in the “providers”<\/span> section for the Linux artifact. When DotSlash fetches an artifact, it will try the providers in order until one succeeds. Regardless of which provider is used, the downloaded binary will be verified against the specified “hash”<\/span>, “digest”<\/span>,\u00a0 and “size”<\/span>\u00a0values.<\/span><\/p>\n

In this case, the first provider is an ordinary, public URL that can be fetched using curl –location<\/span>, but the second is an example of a <\/span>custom provider<\/span> discussed earlier. The “type”: “github-release” <\/span>line indicates that the <\/span>GitHub<\/span> provider for DotSlash should be used, which shells out to the <\/span>GitHub CLI<\/span><\/a> (gh<\/span>, which must be installed separately from DotSlash) to fetch the artifact instead of <\/span>curl<\/span>. Because facebook\/hermes<\/span> is a public GitHub repository, the first provider should be sufficient here. However, if the repository were private and the fetch required authentication, we would expect the first provider to fail and DotSlash would fallback to the GitHub provider. Assuming the user had run gh auth login <\/span>in advance to configure credentials for the specified repo, DotSlash would be able to fetch the artifact using gh release download<\/span>.<\/span><\/p>\n

By publishing DotSlash files as part of GitHub releases, users can copy them to their own repositories to \u201cvendor in\u201d a specific version of your tool with minimal effect on their repository size, regardless of how large your releases might be.<\/span><\/p>\n

Try DotSlash Today\u00a0<\/span><\/h2>\n

Visit the <\/span>DotSlash site for<\/span><\/a> more in-depth documentation and technical details. The site includes instructions on <\/span>Installing DotSlash<\/span><\/a> so you can start playing with it firsthand.\u00a0<\/span><\/p>\n

We also encourage you to <\/span>check out the DotSlash source code<\/span><\/a> and provide feedback via <\/span>GitHub issues<\/span><\/a>. We look forward to hearing from you!<\/span><\/p>\n

The post DotSlash: Simplified executable deployment<\/a> appeared first on Engineering at Meta<\/a>.<\/p>\n

Engineering at Meta<\/p>","protected":false},"excerpt":{"rendered":"

We\u2019ve open sourced DotSlash, a tool that makes large executables available in source control with a negligible impact on repository size, thus avoiding I\/O-heavy clone operations. With DotSlash, a set of platform-specific executables is replaced with a single script containing descriptors for the supported platforms. DotSlash handles transparently fetching, decompressing, and verifying the appropriate remote… Continue reading DotSlash: Simplified executable deployment<\/span><\/a><\/p>\n","protected":false},"author":0,"featured_media":0,"comment_status":"","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"spay_email":"","footnotes":""},"categories":[7],"tags":[],"class_list":["post-820","post","type-post","status-publish","format-standard","hentry","category-technology","entry"],"jetpack_featured_media_url":"","jetpack-related-posts":[{"id":830,"url":"https:\/\/fde.cat\/index.php\/2024\/02\/26\/how-dotslash-makes-executable-deployment-simpler\/","url_meta":{"origin":820,"position":0},"title":"How DotSlash makes executable deployment simpler","date":"February 26, 2024","format":false,"excerpt":"Andres Suarez and Michael Bolin, two software engineers at Meta, join Pascal Hartig (@passy) on the Meta Tech Podcast to discuss the ins and outs of DotSlash, a new open source tool from Meta. DotSlash takes the pain out of distributing binaries and toolchains to developers. Instead of committing large,\u2026","rel":"","context":"In "Technology"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":818,"url":"https:\/\/fde.cat\/index.php\/2024\/01\/29\/improving-machine-learning-iteration-speed-with-faster-application-build-and-packaging\/","url_meta":{"origin":820,"position":1},"title":"Improving machine learning iteration speed with faster application build and packaging","date":"January 29, 2024","format":false,"excerpt":"Slow build times and inefficiencies in packaging and distributing execution files were costing our ML\/AI engineers a significant amount of time while working on our training stack. By addressing these issues head-on, we were able to reduce this overhead by double-digit percentages.\u00a0 In the fast-paced world of AI\/ML development, it\u2019s\u2026","rel":"","context":"In "Technology"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":625,"url":"https:\/\/fde.cat\/index.php\/2022\/08\/30\/hyperpacks-using-buildpacks-to-build-hyperforce\/","url_meta":{"origin":820,"position":2},"title":"Hyperpacks: Using Buildpacks to Build Hyperforce","date":"August 30, 2022","format":false,"excerpt":"At Salesforce we regularly use our products and services to scale our own business. One example is Buildpacks, which we created nearly a decade ago and is now a part of Hyperforce. Hyperpacks are an innovative new way of using Cloud Native Buildpacks (CNB) to manage our public cloud infrastructure.\u00a0\u2026","rel":"","context":"In "Technology"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":606,"url":"https:\/\/fde.cat\/index.php\/2022\/07\/14\/owl-distributing-content-at-meta-scale\/","url_meta":{"origin":820,"position":3},"title":"Owl: Distributing content at Meta scale","date":"July 14, 2022","format":false,"excerpt":"Being able to distribute large, widely -consumed objects (so-called hot content) efficiently to hosts is becoming increasingly important within Meta\u2019s private cloud. These are commonly distributed content types such as executables, code artifacts, AI models, and search indexes that help enable our software systems. Owl is a new system for\u2026","rel":"","context":"In "Technology"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":728,"url":"https:\/\/fde.cat\/index.php\/2023\/06\/27\/meta-developer-tools-working-at-scale\/","url_meta":{"origin":820,"position":4},"title":"Meta developer tools: Working at scale","date":"June 27, 2023","format":false,"excerpt":"Every day, thousands of developers at Meta are working in repositories with millions of files. Those developers need tools that help them at every stage of the workflow while working at extreme scale. In this article we\u2019ll go through a few of the tools in the development process. And, as\u2026","rel":"","context":"In "Technology"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":551,"url":"https:\/\/fde.cat\/index.php\/2022\/03\/10\/code-verify-an-open-source-browser-extension-for-verifying-code-authenticity-on-the-web\/","url_meta":{"origin":820,"position":5},"title":"Code Verify: An open source browser extension for verifying code authenticity on the web","date":"March 10, 2022","format":false,"excerpt":"Since WhatsApp introduced multi-device capability last year, we\u2019ve seen an increase in people accessing WhatsApp directly through their web browser via WhatsApp Web. With this shift in mind, we\u2019ve been looking at ways to add additional layers of security to the WhatsApp Web experience. Starting today, you can now use\u2026","rel":"","context":"In "Technology"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"_links":{"self":[{"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/posts\/820","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/types\/post"}],"replies":[{"embeddable":true,"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/comments?post=820"}],"version-history":[{"count":0,"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/posts\/820\/revisions"}],"wp:attachment":[{"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/media?parent=820"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/categories?post=820"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/tags?post=820"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}