Skip to content

Less frequent progress updates for percentage complete when processing logging #457

@carlhiggs

Description

@carlhiggs

Currently, when running the workflow analysis wrapper function, this writes progress to the log file with an update very frequently, perhaps multiple times per second. If these were printed to screen that would maybe be fine --- like realtime feedback. But printed to a log file it is more than excessive (although, it can be helpful, for tracking pogress).

For example, for very large study regions, the updates cause the log files to be extremely long and take up unnecessary disk space, and unnecessary time making disk writes
image

While an ideal solution might be to implement proper logging functionality, a quick solution that would reduce the size of logging outputs while still allowing tracking of progress would be to reduce the frequency of updates, e.g. so only increments of 1% complete are reported.

Example of how to do this are described here:
https://stackoverflow.com/questions/47995958/python-tqdm-package-how-to-configure-for-less-frequent-status-bar-updates

This isn't a high priority, but it would be nice to have.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions