Uploaded image for project: 'The Virtual Brain'
  1. The Virtual Brain
  2. TVB-2073

simple text format for simulation specification

    XMLWordPrintable

Details

    Description

      It would be useful to be able to specify and run simulations with simple, minimal text files, rather than directly using the full scripting api commands every time.

      I gave some thought to this a while back, gave it a name 'tvb-scripting', and small github repo . This was actually paired with another small repo interfacing with a library for automated computational research replicability called sumatra.

      Now revisiting this as it is apparent that being able to specifying sims with simple text files will make certain things (e.g. large-scale simulation studies) a lot cleaner and easier to set up.

      So: I propose to put some of the tvb-scripting code from the above repo into tvb-library.

      There are basically three components to this:

      a. standard text file format and corresponding parsers
      b. simulation object constructor and runner
      c. command line function that calls b. with c. as an argument

      Obviously we can have parsers for more than one text file format; they just need to be able to evaluate the parameter file to the same intermediate representation used in the sim object constructor.

      Example .json parameter file here, parser+runner here, command line caller here

      These can be redesigned and improved in various ways.

      Firstly I would probably axe the plotting stuff; that's probably overkill.

      Secondly re: text file format -

      I started out doing this with hierarchical .json, as that seemed the best fit to the tvb python api, which specifies sims with about 3-4 (explicit) nested hierarchical levels (deepest levels are e.g. integrator noise, and stimuli). The advantage of this is that you can just construct a lot of the simulator components by just passing the contents of each level as **args directly (see esp. here; very compact indeed)

      However I think actually we want something a bit cleaner than hierarchical .json. The .json files end up being a bit clunky; too many unnecessary brackets and commas and indentations (see e.g. demo examples here).

      I have written an alternative parser for a flatter config/.ini text file format that I think will be a better option. YAML would be another option.

      If people are on board with this addition then I will make a branch and PR on tvb-library, and also make a notebook tutorial on usage.

      Questions:

      • where should it go?
      • have other people used something similar when e.g. doing large PSEs on clusters?
      • are there any similar representations of simulation specifications in tvb-framework?
      • could/should this fit in with edupack?
      • should we be thinking this in relation to serializing simulator objects?
      • (e.g. saving a simulation could spit out a parameter text file)

      Gliffy Diagrams

        Attachments

          Activity

            People

              john.griffiths John griffiths
              john.griffiths John griffiths
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:

                Time Tracking

                  Estimated:
                  Original Estimate - 3 days
                  3d
                  Remaining:
                  Remaining Estimate - 3 days
                  3d
                  Logged:
                  Time Spent - Not Specified
                  Not Specified