Skip to content

Duplicate processing of an entry causes a crash #1433

@8times4

Description

@8times4

Version

v2.46.0

Flavor

CLI (Command-Line Interface)

Platform

Docker / Ubuntu

Export format

JSON

Steps to reproduce

Try exporting with threads, with parallel 5 (also had the same with 10), and eventually it'll simply crash due to duplicate entries.

    docker run --rm -it -v "$PWD":/out tyrrrz/discordchatexporter:stable export \
        -t "$TOKEN" \
        --channel "$channel_id" \
        --include-threads All \
        --utc \
        --parallel 5 \
        --format Json \
        --fuck-russia \
        --respect-rate-limits true

Details

Resolving channel(s)...
Fetching threads...
Fetched 1235 thread(s).
Exporting 1235 channel(s)...

Thread A        ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━   6%
Thread B        ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━   4%
Thread C        ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━   0%
Thread C        ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━   0%


ERROR
System.IO.IOException: The process cannot access the file '/out/Thread C  [123123123].json' because it is being used by another process.
  at Microsoft.Win32.SafeHandles.SafeFileHandle.Init(String path, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize, Int64& fileLength, UnixFileMode& filePermissions)
  at Microsoft.Win32.SafeHandles.SafeFileHandle.Open(String fullPath, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize, UnixFileMode openPermissions, Int64& fileLength, UnixFileMode& filePermissions, Boolean failForSymlink, Boolean& wasSymlink, Func`4 createOpenException)
  at Microsoft.Win32.SafeHandles.SafeFileHandle.Open(String fullPath, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize, Nullable`1 unixCreateMode, Func`4 createOpenException)
  at System.IO.Strategies.OSFileStreamStrategy..ctor(String path, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize, Nullable`1 unixCreateMode)
  at System.IO.Strategies.UnixFileStreamStrategy..ctor(String path, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize, Nullable`1 unixCreateMode)
  at System.IO.Strategies.FileStreamHelpers.ChooseStrategyCore(String path, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize, Nullable`1 unixCreateMode)
  at System.IO.Strategies.FileStreamHelpers.ChooseStrategy(FileStream fileStream, String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options, Int64 preallocationSize, Nullable`1 unixCreateMode)
  at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options, Int64 preallocationSize)
  at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options)
  at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, Boolean useAsync)
  at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize)
  at System.IO.File.Create(String path, Int32 bufferSize)
  at System.IO.File.Create(String path)
  at DiscordChatExporter.Core.Exporting.MessageExporter.CreateMessageWriter(String filePath, ExportFormat format, ExportContext context) in /tmp/app/DiscordChatExporter.Core/Exporting/MessageExporter.cs:115
  at DiscordChatExporter.Core.Exporting.MessageExporter.InitializeWriterAsync(CancellationToken cancellationToken) in /tmp/app/DiscordChatExporter.Core/Exporting/MessageExporter.cs:40
  at DiscordChatExporter.Core.Exporting.MessageExporter.DisposeAsync() in /tmp/app/DiscordChatExporter.Core/Exporting/MessageExporter.cs:77
  at DiscordChatExporter.Core.Exporting.ChannelExporter.ExportChannelAsync(ExportRequest request, IProgress`1 progress, CancellationToken cancellationToken) in /tmp/app/DiscordChatExporter.Core/Exporting/ChannelExporter.cs:68
  at DiscordChatExporter.Cli.Commands.Base.ExportCommandBase.<>c__DisplayClass67_3.<<ExportAsync>b__4>d.MoveNext()
  at DiscordChatExporter.Cli.Utils.Extensions.ConsoleExtensions.StartTaskAsync(ProgressContext context, String description, Func`2 performOperationAsync)
  at DiscordChatExporter.Cli.Commands.Base.ExportCommandBase.<>c__DisplayClass67_2.<<ExportAsync>b__3>d.MoveNext()
  at System.Threading.Tasks.Parallel.<>c__53`1.<<ForEachAsync>b__53_0>d.MoveNext()
  at DiscordChatExporter.Cli.Commands.Base.ExportCommandBase.<>c__DisplayClass67_0.<<ExportAsync>b__0>d.MoveNext()
  at Spectre.Console.Progress.<>c__DisplayClass31_0.<<StartAsync>b__0>d.MoveNext() in /_/src/Spectre.Console/Live/Progress/Progress.cs:103
  at Spectre.Console.Progress.<>c__DisplayClass32_0`1.<<StartAsync>b__0>d.MoveNext() in /_/src/Spectre.Console/Live/Progress/Progress.cs:138
  at Spectre.Console.Internal.DefaultExclusivityMode.RunAsync[T](Func`1 func) in /_/src/Spectre.Console/Internal/DefaultExclusivityMode.cs:40
  at Spectre.Console.Progress.StartAsync[T](Func`2 action) in /_/src/Spectre.Console/Live/Progress/Progress.cs:121
  at Spectre.Console.Progress.StartAsync(Func`2 action) in /_/src/Spectre.Console/Live/Progress/Progress.cs:101
  at DiscordChatExporter.Cli.Commands.Base.ExportCommandBase.ExportAsync(IConsole console, IReadOnlyList`1 channels)
  at DiscordChatExporter.Cli.Commands.ExportChannelsCommand.ExecuteAsync(IConsole console)
  at CliFx.CliApplication.RunAsync(ApplicationSchema applicationSchema, CommandInput commandInput) in /_/CliFx/CliApplication.cs:144
  at CliFx.CliApplication.RunAsync(IReadOnlyList`1 commandLineArguments, IReadOnlyDictionary`2 environmentVariables) in /_/CliFx/CliApplication.cs:185

Checklist

  • I have looked through existing issues to make sure that this bug has not been reported before
  • I have provided a descriptive title for this issue
  • I have made sure that this bug is reproducible on the latest version of the application
  • I have provided all the information needed to reproduce this bug as efficiently as possible
  • I have sponsored this project
  • I have not read any of the above and just checked all the boxes to submit the issue

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions