Argthtjtr

OK, I have a question about how to lay out code efficiently.

I have a model written in python which generates results which I use to produce graphs in matplotlib. As written, the model is contained within a single file, and I have 15 other run-files, which call on it with complicated configurations and produce graphs. It takes a while to go through and run each of these run-files, but since they all use substantially different settings for the model, I need to have complicated setup files anyway, and it all works.

I have the output set up for figures which could go in an academic paper. I have now realised that I am going to need each of these figures again in other formats - one for presentations (low dpi, medium size, different font) and one for a poster (high dpi, much bigger, different font again.)

This means I could potentially have 45 odd files to wade through every time I want to make a change to my model. I also would have to cut and paste a lot of boilerplate matplotlib code with minor alterations (each run-file would become 3 different files - one for each graph).

Can anybody explain to me how (and if) I could speed things up? At the moment, I think it's taking me much longer than it should.

As I see it there are 3 main options:

1. Set up 3 run-files for each actual model run (so duplicate a fair amount, and run the model a lot more than I need) but I can then tweak everything independently (but risk missing something important).


2. Add another layer - so save the results as .csv or equivalent and then read them into the files for producing graphs. This means more files, but I only have to run the model once per 3 graphs (which might save some time).


3. Keep the graph and model parameter files integrated, but add another file which sets up graphing templates, so every time I run the file it spits out 3 graphs) It might speed things up a bit, and will certainly keep the number of files down, but they will get very big (and probably much more complicated).


4. Something else..

Can anybody point me to a resource or provide me with some advice on how best to handle this?

Thanks!

I think you are close to find what you want.

If calculations take some time, store results in files to process later without recalculation.

The most important: separate code from configuration, instead of copy pasting variations of such mixture.

If the model takes parameters, define a model class. Maybe instantiate the model only once, but the model knows how to load_config, read_input_data and run. Model also does write_results. That way you can loop a sequence of load_config, read_data, write_results for every config and maybe input data.

Write the config files by hand with ini format for example and use the confiparser module to load them.

Do something similar for your Graph class. Put the template definition in configuration files, including output format, sizes fonts, and so on.

In the end you will be able to "manage" the intended workflow with a single script that uses this facilites. Maybe store groups of related configuration files, output templates and input data together, one group per folder for each modelling session.