Speedup scons over your small project

TL;DR

In your SConstruct file, add the tools keyword argument to Environment and DefaultEnvironment initialization to explicitly specify the tools needed in your project. It saves a lot of time.

The Story

My resume was written in LaTeX and convert to PDF using a handwritten Makefile, of which I always want to replace using a more advanced automation tool.

So I'm looking at scons. It's written in Python which is a language I like, and looks much simpler than CMake. But one thing I don't like is the speed.

Here's the original SConstruct:

convert = Builder(action=[
    "convert -alpha off -density 300 $SOURCE -append $TARGET"])

env = Environment(BUILDERS={"Convert": convert})

pdf = env.PDF("resume.tex")
png = env.Convert('resume.png', pdf)
Default(pdf)

So what this SConstruct file did is that it compiles the resume.tex file into PDF, and then optionally convert the PDF into a PNG file.

And the time it takes? Here it is:

$ scons -f SConstruct.before --debug=time . |grep ^Total
Total build time: 8.844431 seconds
Total SConscript file execution time: 2.516214 seconds
Total SCons execution time: 0.094147 seconds
Total command execution time: 6.234070 seconds

In the total build time ~8 seconds, ~3 seconds was used in executing the SConstruct file. Let's run a cleanup which make it more obvious:

$ scons -f SConstruct.before --debug=time -c . |grep ^Total
Total build time: 2.669596 seconds
Total SConscript file execution time: 2.579540 seconds
Total SCons execution time: 0.090056 seconds
Total command execution time: 0.000000 seconds

In a total build time ~2.6 seconds, ~2.5 seconds is wasted.

I know that there's a lot posts talking about how slow scons is but I don't know it could be that slow. If this is a large project managed by scons, 2.5 seconds seems insignificant but for a simple task like compiling LaTeX file it's unacceptable.

After a simple profiling it become obvious that scons takes too much time in initializing the Environment instance:

$ python2 -m cProfile -s cumtime `which scons` -f SConstruct.before -c .
...
   Ordered by: cumulative time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
...
        2    0.000    0.000    2.838    1.419 Environment.py:917(__init__)

Luckily there's a GoFastButton page on scons wiki that caught my eyes: too many unnessesary tools in the Environment is a cause of slow down.

It's straight forward after figuring out the cause. Here's an updated SConstruct:

DefaultEnvironment(tools=[])
convert = Builder(action=[
    "convert -alpha off -density 300 $SOURCE -append $TARGET"])

env = Environment(BUILDERS={"Convert": convert}, tools=['pdftex'])

pdf = env.PDF("resume.tex")
png = env.Convert('resume.png', pdf) 
Default(pdf)

Notice the tools keyword argument in initializing Environment, and the tools keyword argument to DefaultEnvironment. The result is a significantly reduced build time (for a small project):

$ scons -f SConstruct.after --debug time . |grep ^Total
Total build time: 6.579649 seconds
Total SConscript file execution time: 0.037326 seconds
Total SCons execution time: 0.112706 seconds
Total command execution time: 6.429617 seconds

$ scons -f SConstruct.after --debug time . -c |grep ^Total
Total build time: 0.122485 seconds
Total SConscript file execution time: 0.030382 seconds
Total SCons execution time: 0.092103 seconds
Total command execution time: 0.000000 seconds

Conclusion?

If you know what you are going to build (you already know it would be a C++ project, or in my case building PDF from LaTeX), you'd better tell scons explicit what tools you want to be included in the Environment. If your project is small, the saved time would be significant.

P.S. I found it hard to decide what tools you want to be included in Environment. The full list of available tools can be found in man 1 scons, but there's no description on what those tools do. For example, in my case of compiling *.tex file into *.pdf file, there are three relevant tools available: latex, pdftex, pdflatex. I have to test among them to know pdflatex is the right one.

Comments !

blogroll

social