Performance problem with PyYAML

Sebastian Huber sebastian.huber at embedded-brains.de
Fri Nov 8 10:33:53 UTC 2019


I experimented a bit with marshal, pickle, and ast.literal_eval. I think 
that using a cache file is the way to go. An update evaluation based on 
the mtime is quite fast. You need the YAML to X conversion only if a 
build specification file changed or upon first use. We don't have to 
store the cache file in the repository (could be 1MiB or more). See 
attached example scripts.

My favourite is a conversion to pickle. It is supposed to be a stable 
format and was also the fastest, about 0.1 seconds load time.

-- 
Sebastian Huber, embedded brains GmbH

Address : Dornierstr. 4, D-82178 Puchheim, Germany
Phone   : +49 89 189 47 41-16
Fax     : +49 89 189 47 41-09
E-Mail  : sebastian.huber at embedded-brains.de
PGP     : Public key available on request.

Diese Nachricht ist keine geschäftliche Mitteilung im Sinne des EHUG.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: yaml2ast.py
Type: text/x-python
Size: 2677 bytes
Desc: not available
URL: <http://lists.rtems.org/pipermail/devel/attachments/20191108/a49690ab/attachment-0003.py>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: yaml2marshal.py
Type: text/x-python
Size: 2657 bytes
Desc: not available
URL: <http://lists.rtems.org/pipermail/devel/attachments/20191108/a49690ab/attachment-0004.py>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: yaml2pickle.py
Type: text/x-python
Size: 2653 bytes
Desc: not available
URL: <http://lists.rtems.org/pipermail/devel/attachments/20191108/a49690ab/attachment-0005.py>


More information about the devel mailing list