Skip to content

Add makedirs to FileUtil, move where the directory templates are created #2004

@RhetTbull

Description

@RhetTbull
  • Error: The SMB mounted folder "disappears" from mcOS and osxphotos gets "Errno 13" -- yeah! Have to figure out why after 22 hours straight working well and osxphotos exporting ~75K assets, the connection fails.
2025-11-13 16:15:39.003688 -- Exporting IMG_20240205_130446.HEIC (0BB4248D-DB86-4E67-BC3E-364E32EC4E4D.heic) (76594/112517)
2025-11-13 16:17:25.471106 -- Error exporting photo (0BB4248D-DB86-4E67-BC3E-364E32EC4E4D: IMG_20240205_130446.HEIC) as IMG_20240205_130446.jpeg: [Errno 57] Socket is not connected: '/Volumes/photo/xxx.HEIC'
2025-11-13 16:17:25.486767 -- Retrying export for photo (0BB4248D-DB86-4E67-BC3E-364E32EC4E4D: IMG_20240205_130446.HEIC)
2025-11-13 16:17:25.498956 -- Error exporting photo (0BB4248D-DB86-4E67-BC3E-364E32EC4E4D: IMG_20240205_130446.HEIC) as IMG_20240205_130446.jpeg: Invalid path passed to export
2025-11-13 16:17:25.499440 -- Retrying export for photo (0BB4248D-DB86-4E67-BC3E-364E32EC4E4D: IMG_20240205_130446.HEIC)
2025-11-13 16:17:25.502350 -- Error exporting photo (0BB4248D-DB86-4E67-BC3E-364E32EC4E4D: IMG_20240205_130446.HEIC) as IMG_20240205_130446.jpeg: Invalid path passed to export
Something went wrong and osxphotos encountered an error:
[Errno 13] Permission denied: '/Volumes/photo'
Writing export database to 
/Users/msp/Pictures/Pictures/Logs/.osxphotos_export.db
Crash log written to '/Users/msp/Pictures/Pictures/Apps/osxphotos_crash.log'
Please file a bug report at https://github.com/RhetTbull/osxphotos/issues with 
the crash log attached.
2025-11-13 16:17:27.224896 -- Cleaning up lock files
  • Yes, I use --ramdb and its copying to a NAS Server.

  • I also use the --report which yields the following counts which seem to be correct.

report.csv counts
[
{
  "exported_sum": 75462,
  "new_sum": 15144,
  "updated_sum": 60318,
  "skipped_sum": 2698,
  "exif_updated_sum": 75457,
  "touched_sum": 75462,
  "converted_to_jpeg_sum": 12654,
  "missing_sum": 4,
  "error_sum": 0,
  "exiftool_warning_sum": 0,
  "exiftool_error_sum": 0,
  "cleanup_deleted_file_sum": 0,
  "cleanup_deleted_directory_sum": 0,
  "exported_album_sum": 0
}
]
  • osxphotos_crash.log listed here
osxphotos_crash.log
osxphotos crash log
Created: 2025-11-13 16:17:26.933997
osxphotos version: 0.74.0
Platform: macOS-15.7.1-arm64-arm-64bit-Mach-O
Python version: 3.13.8 (main, Oct  7 2025, 12:01:51) [Clang 17.0.0 (clang-1700.0.13.3)]
sys.argv: ['/Users/msp/Pictures/Pictures/Apps/.venv/bin/osxphotos', 'export', '/Volumes/photo/Family.Photos', '--exportdb', '/Users/msp/Pictures/Pictures/Logs/.osxphotos_export.db', '--library', '/Users/msp/Pictures/Recreate.photoslibrary/database/Photos.sqlite', '--load-config', '/Users/msp/Pictures/Pictures/Apps/colibri.toml', '--report', '/Users/msp/Pictures/Pictures/Logs/report.csv']
CRASH_DATA:
locals: {'dest': '/Volumes/photo/Family.Photos', 'db': '/Users/msp/Pictures/Recreate.photoslibrary/database/Photos.sqlite', 'add_exported_to_album': None, 'add_missing_to_album': None, 'add_skipped_to_album': None, 'added_after': None, 'added_before': None, 'added_in_last': None, 'album': (), 'album_keyword': False, 'alt_db': None, 'alt_copy': False, 'append': False, 'fix_orientation': False, 'beta': False, 'burst': False, 'checkpoint': None, 'cleanup': False, 'cloudasset': False, 'config_only': False, 'convert_to_jpeg': False, 'crash_after': None, 'current_name': False, 'deleted': False, 'deleted_only': False, 'description': (), 'description_template': None, 'directory': None, 'download_missing': False, 'dry_run': False, 'duplicate': False, 'edited': False, 'edited_suffix': None, 'exif': (), 'exiftool': False, 'exiftool_merge_keywords': False, 'exiftool_merge_persons': False, 'exiftool_option': (), 'exiftool_path': None, 'export_as_hardlink': False, 'export_by_date': False, 'exportdb': '/Users/msp/Pictures/Pictures/Logs/.osxphotos_export.db', 'external_edit': False, 'favorite': False, 'favorite_rating': False, 'filename_template': None, 'finder_tag_keywords': False, 'finder_tag_template': (), 'folder': (), 'force_update': False, 'from_date': None, 'from_time': None, 'has_comment': False, 'has_likes': False, 'has_raw': False, 'hdr': False, 'hidden': False, 'ignore_case': False, 'ignore_date_modified': False, 'ignore_exportdb': False, 'ignore_signature': False, 'in_album': False, 'incloud': False, 'is_reference': False, 'jpeg_ext': None, 'jpeg_quality': None, 'keep': (), 'keyword': (), 'keyword_template': (), 'label': (), 'limit': None, 'live': False, 'load_config': '/Users/msp/Pictures/Pictures/Apps/colibri.toml', 'location': False, 'max_size': None, 'min_size': None, 'missing': False, 'name': (), 'no_comment': False, 'no_description': False, 'no_exportdb': False, 'no_keyword': False, 'no_likes': False, 'no_location': False, 'no_place': False, 'no_progress': False, 'no_title': False, 'not_burst': False, 'not_cloudasset': False, 'not_edited': False, 'not_favorite': False, 'not_hdr': False, 'not_hidden': False, 'not_in_album': False, 'not_incloud': False, 'not_live': False, 'not_missing': False, 'not_panorama': False, 'not_portrait': False, 'not_reference': False, 'not_screenshot': False, 'not_screen_recording': False, 'not_selfie': False, 'not_shared': False, 'not_slow_mo': False, 'not_time_lapse': False, 'only_movies': False, 'only_new': False, 'only_photos': False, 'original_suffix': None, 'overwrite': False, 'panorama': False, 'person': (), 'person_keyword': False, 'place': (), 'portrait': False, 'post_command': (), 'post_command_error': None, 'post_function': (), 'preview': False, 'preview_if_missing': False, 'preview_suffix': None, 'print_template': (), 'query_eval': (), 'query_function': (), 'ramdb': False, 'regex': (), 'replace_keywords': False, 'report': '/Users/msp/Pictures/Pictures/Logs/report.csv', 'retry': None, 'save_config': None, 'screenshot': False, 'screen_recording': False, 'selfie': False, 'shared': False, 'export_aae': False, 'sidecar': (), 'sidecar_drop_ext': False, 'sidecar_template': (), 'skip_bursts': False, 'skip_edited': False, 'skip_live': False, 'skip_original_if_edited': False, 'skip_raw': False, 'skip_uuid': (), 'skip_uuid_from_file': None, 'slow_mo': False, 'strip': False, 'theme': None, 'time_lapse': False, 'timestamp': False, 'title': (), 'tmpdir': None, 'to_date': None, 'to_time': None, 'touch_file': False, 'update': False, 'update_errors': False, 'use_photokit': False, 'use_photos_export': False, 'uti': None, 'uuid': (), 'uuid_from_file': None, 'verbose_flag': 0, 'xattr_template': (), 'year': (), 'syndicated': False, 'not_syndicated': False, 'saved_to_library': False, 'not_saved_to_library': False, 'shared_moment': False, 'not_shared_moment': False, 'shared_library': False, 'not_shared_library': False, 'selected': False, 'ctx': <click.core.Context object at 0x11343ae90>, 'cli_obj': <osxphotos.cli.cli.CLI_Obj object at 0x1134d6120>}
cfg: {'add_exported_to_album': None, 'add_missing_to_album': None, 'add_skipped_to_album': None, 'added_after': None, 'added_before': None, 'added_in_last': None, 'album': (), 'album_keyword': False, 'alt_copy': False, 'alt_db': None, 'append': False, 'beta': False, 'burst': False, 'checkpoint': None, 'cleanup': True, 'cloudasset': False, 'convert_to_jpeg': True, 'crash_after': None, 'current_name': False, 'db': '/Users/msp/Pictures/Recreate.photoslibrary/database/Photos.sqlite', 'deleted': False, 'deleted_only': False, 'description': (), 'description_template': 'Album:{album|sort,}{newline}Description:{descr,}', 'directory': '{album|filter(startswith 1|2)?{album|function:tree.py::determine_folder_name},0000-0000}/{album|filter(startswith 1|2)[/,.|:,.]}', 'download_missing': True, 'dry_run': False, 'duplicate': False, 'edited': False, 'edited_suffix': None, 'exif': (), 'exiftool': True, 'exiftool_merge_keywords': True, 'exiftool_merge_persons': True, 'exiftool_option': (), 'exiftool_path': None, 'export_aae': False, 'export_as_hardlink': False, 'export_by_date': False, 'exportdb': '/Users/msp/Pictures/Pictures/Logs/.osxphotos_export.db', 'external_edit': False, 'favorite': False, 'favorite_rating': False, 'filename_template': None, 'finder_tag_keywords': False, 'finder_tag_template': (), 'fix_orientation': False, 'folder': ('iPhoto Events', 'Family'), 'force_update': False, 'from_date': None, 'from_time': None, 'has_comment': False, 'has_likes': False, 'has_raw': False, 'hdr': False, 'hidden': False, 'ignore_case': False, 'ignore_date_modified': False, 'ignore_exportdb': True, 'ignore_signature': False, 'in_album': False, 'incloud': False, 'is_reference': False, 'jpeg_ext': None, 'jpeg_quality': None, 'keep': ('index.png', '[0-9][0-9][0-9][0-9]-[0-9][0-9][0-9][0-9]/[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9].png'), 'keyword': (), 'keyword_template': ('{keyword}', '{label}', '{searchinfo.activity?activity:{searchinfo.activity},}', '{searchinfo.venue_type?venue_type:{searchinfo.venue_type},}', '{person|sort|function:persons.py::myfilter}', '{function:media_keywords.py::media_keywords}'), 'label': (), 'library': None, 'limit': None, 'live': False, 'location': False, 'max_size': None, 'min_size': None, 'missing': False, 'name': (), 'no_comment': False, 'no_description': False, 'no_exportdb': False, 'no_keyword': False, 'no_likes': False, 'no_location': False, 'no_place': False, 'no_progress': True, 'no_title': False, 'not_burst': False, 'not_cloudasset': False, 'not_edited': False, 'not_favorite': False, 'not_hdr': False, 'not_hidden': True, 'not_in_album': False, 'not_incloud': False, 'not_live': False, 'not_missing': False, 'not_panorama': False, 'not_portrait': False, 'not_reference': False, 'not_saved_to_library': False, 'not_screen_recording': False, 'not_screenshot': False, 'not_selfie': False, 'not_shared': False, 'not_shared_library': False, 'not_shared_moment': False, 'not_slow_mo': False, 'not_syndicated': False, 'not_time_lapse': False, 'only_movies': False, 'only_new': False, 'only_photos': False, 'original_suffix': None, 'overwrite': False, 'panorama': False, 'person': (), 'person_keyword': False, 'place': (), 'portrait': False, 'post_command': (), 'post_command_error': None, 'post_function': (), 'preview': False, 'preview_if_missing': False, 'preview_suffix': None, 'print_template': (), 'query_eval': (), 'query_function': (), 'ramdb': True, 'regex': (), 'replace_keywords': False, 'report': '/Users/msp/Pictures/Pictures/Logs/report.csv', 'retry': 2, 'saved_to_library': False, 'screen_recording': False, 'screenshot': False, 'selected': False, 'selfie': False, 'shared': False, 'shared_library': False, 'shared_moment': False, 'sidecar': (), 'sidecar_drop_ext': False, 'sidecar_template': (), 'skip_bursts': True, 'skip_edited': False, 'skip_live': False, 'skip_original_if_edited': True, 'skip_raw': True, 'skip_uuid': (), 'skip_uuid_from_file': None, 'slow_mo': False, 'strip': True, 'syndicated': False, 'theme': None, 'time_lapse': False, 'timestamp': True, 'title': (), 'tmpdir': None, 'to_date': None, 'to_time': None, 'touch_file': True, 'update': True, 'update_errors': False, 'use_photokit': False, 'use_photos_export': False, 'uti': None, 'uuid': (), 'uuid_from_file': None, 'verbose': True, 'xattr_template': (), 'year': ()}
osxphotos version: 0.74.0
Error: [Errno 13] Permission denied: '/Volumes/photo'
Traceback (most recent call last):
  File "/Users/msp/Pictures/Pictures/Apps/.venv/lib/python3.13/site-packages/osxphotos/crash_reporter.py", line 79, in wrapped
    return func(*args, **kwargs)
  File "/Users/msp/Pictures/Pictures/Apps/.venv/lib/python3.13/site-packages/osxphotos/cli/export.py", line 1077, in export
    return_code = export_cli(**locals())
  File "/Users/msp/Pictures/Pictures/Apps/.venv/lib/python3.13/site-packages/osxphotos/cli/export.py", line 1922, in export_cli
    export_results = export_photo(**kwargs)
  File "/Users/msp/Pictures/Pictures/Apps/.venv/lib/python3.13/site-packages/osxphotos/cli/export.py", line 2381, in export_photo
    dest_paths = get_dirnames_from_template(
        photo,
    ...<6 lines>...
        export_db=export_db,
    )
  File "/Users/msp/Pictures/Pictures/Apps/.venv/lib/python3.13/site-packages/osxphotos/cli/export.py", line 2931, in get_dirnames_from_template
    os.makedirs(dest_path)
    ~~~~~~~~~~~^^^^^^^^^^^
  File "<frozen os>", line 218, in makedirs
  File "<frozen os>", line 218, in makedirs
  File "<frozen os>", line 218, in makedirs
  File "<frozen os>", line 228, in makedirs
PermissionError: [Errno 13] Permission denied: '/Volumes/photo'
  • Before I go and run it again, I need to make sure it actually skips already exported pics.

  • sqlite3 .osxphotos_export.db is showing me:

sqlite> select count(*) from export_data;
count(*)
--------
107813  
sqlite> select count(*) from photoinfo;
count(*)
--------
105659  
  • And then I went into the history table.. and it seems the files are being savagely updated in Photos thus generating diffs and new signatures (?)... which prompts osxphotos (correctly!) to (re)exports. And the odd skip entry shows up... Will it ever stabilise, as I'm running after my first export in five days!!! -- I guess, back to the first bullet: figure out why the SMB Mount connects drops.
    • Idea/thought: will double-check the retry timeout to write to a file to see if a larger one would allow me somehow to re-mount the SMDB filesystem... avoiding the restart of osxphotos export
 select e.id, e.uuid,  dest_mtime, timestamp, error, h.datetime, h.action, h.diff from export_data e, history h, history_path hp where h.filepath_id = hp.id and  e.uuid = hp.uuid  group by e.id, e.filepath_normalized, e.uuid,  dest_mtime, digest, timestamp, error, h.datetime, h.action order by hp.uuid limit 10
   ...> ;
id     uuid                                  dest_mtime    timestamp                error   datetime                 action                      diff                                 
-----  ------------------------------------  ------------  -----------------------  ------  -----------------------  --------------------------  -------------------------------------
48694  000038D2-3B1B-4C5D-8767-B9C9AA6D6DF5  1310998675.0  2025-11-13 08:13:51.888  <NULL>  2025-11-09 23:29:59.713  new                         <NULL>                               
48694  000038D2-3B1B-4C5D-8767-B9C9AA6D6DF5  1310998675.0  2025-11-13 08:13:51.888  <NULL>  2025-11-12 01:18:16.964  update: DEST_SIG_DIFFERENT  [["adjustment_type", "removed", [0]]]
48694  000038D2-3B1B-4C5D-8767-B9C9AA6D6DF5  1310998675.0  2025-11-13 08:13:51.888  <NULL>  2025-11-13 08:13:51.888  skip                        [["adjustment_type", "removed", [0]]]
55986  0000BA18-134D-4B8E-90DF-C3E059A247BA  1334474471.0  2025-11-12 22:27:12.732  <NULL>  2025-11-10 03:53:01.583  new                         <NULL>                               
55986  0000BA18-134D-4B8E-90DF-C3E059A247BA  1334474471.0  2025-11-12 22:27:12.732  <NULL>  2025-11-12 22:27:12.732  update: DEST_SIG_DIFFERENT  [["adjustment_type", "removed", [0]]]
20814  0000F631-B34D-41EF-8EAF-89A6E8D4B18B  1679225364.0  2025-11-09 07:06:41.299  <NULL>  2025-11-09 07:06:41.299  new                         <NULL>                               
73593  000126EB-BC9D-4E20-80DB-328C74874527  1714734902.0  2025-11-12 20:39:38.180  <NULL>  2025-11-10 22:38:44.874  new                         <NULL>                               
73593  000126EB-BC9D-4E20-80DB-328C74874527  1714734902.0  2025-11-12 20:39:38.180  <NULL>  2025-11-10 22:38:45.178  new                         [["adjustment_type", "removed", [0]]]
73593  000126EB-BC9D-4E20-80DB-328C74874527  1714734902.0  2025-11-12 20:39:38.180  <NULL>  2025-11-11 20:13:41.023  update: DEST_SIG_DIFFERENT  [["adjustment_type", "removed", [0]]]
73593  000126EB-BC9D-4E20-80DB-328C74874527  1714734902.0  2025-11-12 20:39:38.180  <NULL>  2025-11-11 20:13:41.405  update: DEST_SIG_DIFFERENT  [["adjustment_type", "removed", [0]]]

Originally posted by @oPromessa in #2002 (reply in thread)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions