Skip to content

Reduce executable size bloat (currently 20MB) #364

Open
@awakecoding

Description

@awakecoding

The current executable size is about 20MB on Windows, and slightly smaller on other platforms. While it is not a deal breaker, it wouldn't hurt to review the build process to see if the executable size could be reduced with compilation options.

  • Look for common build options (optimize for size, trim debug symbols)
  • Look at the list of dependencies and see which ones contribute to the bloat

Activity

added
area/CICDCI/CD | Packaging for various operations systems | Build
and removed
needs triageWaiting for discussion / prioritization by team
on Oct 13, 2020
dopey

dopey commented on Oct 13, 2020

@dopey
Contributor

To follow up here - we don't have any immediate plans to work on reducing the size of the executable. However, the build process is all committed in Makefiles in this repo, so if anyone from the community would like to open a PR, we'd happily take the submission.

Glandos

Glandos commented on Mar 4, 2022

@Glandos

I have some small tests results.
First, sizes:

.rwxr-xr-x  31M user 2022-03-02 00:34 step-cli
.rwxr-xr-x  28M user 2022-03-04 09:48 step-cli.stripped
.rwxr-xr-x 6,4M user 2022-03-04 09:43 step-cli.stripped.upx
.rwxr-xr-x  12M user 2022-03-04 10:44 step-cli.stripped.upx1
.rwxr-xr-x 6,4M user 2022-03-04 09:43 step-cli.stripped.xz

The first on is the binary extracted from .deb.
Second one is stripped with strip -s.
Third one is upx --ultra-brute on the stripped version that took 10 minutes (yes, that's very long).
Fourth one is upx -1 on the stripped version that took less than 2 seconds.
Fifth one is xz -9 on the stripped version.

Now for launch timing:

/tmp ❯ hyperfine "./step-cli version"
Benchmark #1: ./step-cli version
  Time (mean ± σ):      18.3 ms ±   2.0 ms    [User: 15.6 ms, System: 9.0 ms]
  Range (min … max):    14.8 ms …  25.3 ms    135 runs
 
/tmp ❯ hyperfine "./step-cli.stripped version"
Benchmark #1: ./step-cli.stripped version
  Time (mean ± σ):      18.3 ms ±   1.9 ms    [User: 16.3 ms, System: 8.5 ms]
  Range (min … max):    14.4 ms …  24.0 ms    160 runs
 
/tmp ❯ hyperfine "./step-cli.stripped.upx version"
Benchmark #1: ./step-cli.stripped.upx version
  Time (mean ± σ):     523.7 ms ±  51.3 ms    [User: 519.2 ms, System: 8.0 ms]
  Range (min … max):   446.8 ms … 645.9 ms    10 runs

/tmp ❯ hyperfine "./step-cli.stripped.upx1 version" --warmup 1
Benchmark #1: ./step-cli.stripped.upx1 version
  Time (mean ± σ):     127.5 ms ±   7.9 ms    [User: 125.3 ms, System: 6.7 ms]
  Range (min … max):   121.2 ms … 147.7 ms    24 runs

So the result is that if RPM and DEB are compressed with XZ, this is fine. The size on disk will be a little bit bigger, but it's usually harmless.
I've noticed that the current DEB version is using gzip, that should be changed.
You should at least strip binary to have a quick 10% on size. But then, debugging is harder.

UPX can be used, but it really slows down binary start.

added
needs triageWaiting for discussion / prioritization by team
on Mar 4, 2022
dopey

dopey commented on Mar 10, 2022

@dopey
Contributor

Hey, thanks for posting this @Glandos! It's great to have some concrete numbers.

So we are aware that the executable size is relatively large for cli. However, that comes with the territory - statically linked golang binaries. We could strip it down but we'd be losing the debugging facilities you mentioned, and we'd prefer not to do that - at least not by default. We'd be happy to have a target added to the makefile that could build a reduced size executable.

Furthermore, we'd like to understand the case better (beyond just smaller is better).

  • Is this for a constrained device? If so then how much space are we actually working with?
  • Should we be considering removing functional dependencies for facets of the cli that are uncommonly used. For example, if you're not planning on using ssh we could remove those libraries, or libraries for OTT, etc.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/CICDCI/CD | Packaging for various operations systems | Buildenhancement

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

      Development

      No branches or pull requests

        Participants

        @awakecoding@Glandos@dopey

        Issue actions

          Reduce executable size bloat (currently 20MB) · Issue #364 · smallstep/cli