System.ArgumentOutOfRangeException: Index and length must refer to a location within the string

[pinntokuru]

Version

v2.44.1

Flavor

CLI (Command-Line Interface)

Platform

Windows 11

Export format

JSON

Steps to reproduce

The server (invite: https://discord.gg/nhqjydaR8j) is for Japanese language learning and requires the user to pass Japanese tests to view the threads. Here's a screenshot of the thread.

Command Run:

$TOKEN = "TokenHere"
$SERVERID = "617136488840429598"
$EXPORTFORMAT = "Json"
$EXPORTDIRECTORY = "Z:\Discord-Backups\exports\TMW\%b---00-00-00\"
$MEDIADIR = "Z:\Discord-Backups\exports\TMW\_media\" 

DiscordChatExporter.Cli.exe exportguild -t $TOKEN -g $SERVERID -f $EXPORTFORMAT  --after "2023-05-22" -o $EXPORTDIRECTORY --before "2023-05-29" --media --reuse-media --markdown false --include-threads all --include-vc false --media-dir $MEDIADIR --filter ~has:invite

Things that might be causing the error:

• The link to a channel that the user does not have access to (the No Access link)

• The link to a user that has left the server (Xelieu in this case)

Details

Error occurs for a specific thread, in https://discord.com/channels/617136488840429598/1019998042654511106/threads/1111132945210691597.

I have tried running multiple times from multiple machines, it always fails the same way at the same thread.

The same command has run successfully so far for other before/after dates that do not include this thread.

Unhandled exception. System.ArgumentOutOfRangeException: Index and length must refer to a location within the string. (Parameter 'length')
   at System.String.ThrowSubstringArgumentOutOfRange(Int32 startIndex, Int32 length)
   at System.String.Substring(Int32 startIndex, Int32 length)
   at Spectre.Console.Rendering.Segment.SplitOverflow(Segment segment, Nullable`1 overflow, Int32 maxWidth) in /_/src/Spectre.Console/Rendering/Segment.cs:line 365
   at Spectre.Console.Paragraph.SplitLines(Int32 maxWidth) in /_/src/Spectre.Console/Widgets/Paragraph.cs:line 232
   at Spectre.Console.Paragraph.Render(RenderOptions options, Int32 maxWidth) in /_/src/Spectre.Console/Widgets/Paragraph.cs:line 141
   at Spectre.Console.TableRenderer.Render(TableRendererContext context, List`1 columnWidths) in /_/src/Spectre.Console/Widgets/Table/TableRenderer.cs:line 11
   at Spectre.Console.TableRenderer.Render(TableRendererContext context, List`1 columnWidths) in /_/src/Spectre.Console/Widgets/Table/TableRenderer.cs:line 31
   at Spectre.Console.Padder.Render(RenderOptions options, Int32 maxWidth) in /_/src/Spectre.Console/Widgets/Padder.cs:line 45
   at Spectre.Console.Rendering.LiveRenderable.Render(RenderOptions options, Int32 maxWidth)+MoveNext()
   at System.Collections.Generic.List`1.AddRange(IEnumerable`1 collection)
   at Spectre.Console.RenderableExtensions.GetSegments(IAnsiConsole console, RenderOptions options, IEnumerable`1 renderables) in /_/src/Spectre.Console/Extensions/RenderableExtensions.cs:line 37
   at Spectre.Console.AnsiBuilder.Build(IAnsiConsole console, IRenderable renderable) in /_/src/Spectre.Console/Internal/Backends/Ansi/AnsiBuilder.cs:line 17
   at Spectre.Console.AnsiConsoleFacade.Write(IRenderable renderable) in /_/src/Spectre.Console/Internal/Backends/AnsiConsoleFacade.cs:line 38
   at Spectre.Console.ProgressRefreshThread.Run() in /_/src/Spectre.Console/Live/Progress/ProgressRefreshThread.cs:line 45
   at System.Threading.Thread.StartCallback()

Checklist

• I have looked through existing issues to make sure that this bug has not been reported before
• I have provided a descriptive title for this issue
• I have made sure that this bug is reproducible on the latest version of the application
• I have provided all the information needed to reproduce this bug as efficiently as possible
• I have sponsored this project
• I have not read any of the above and just checked all the boxes to submit the issue

[Tyrrrz]

Looks like a Spectre.Console rendering issue. My guess is that it has something to do with the name of the server 🤔

Which terminal are you running the CLI in?

[pinntokuru]

Which terminal are you running the CLI in?

It's Powershell 7.5.0. Running $PSVersionTable gives further details:

Name                           Value
----                           -----
PSVersion                      7.5.0
PSEdition                      Core
GitCommitId                    7.5.0
OS                             Microsoft Windows 10.0.22631
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

[Tyrrrz]

Can you try running the command through a different terminal, out of curiosity?

[pinntokuru]

Running from cmd also resulted in the exact same error for the exact same thread.

[Tyrrrz]

So I tried exporting just the thread you mentioned and it was successful:

Command:

export -c 1111132945210691597 -t ... --media --reuse-media

Also tried in a few other terminals, couldn't reproduce the issue.

[pinntokuru]

So I tried exporting just the thread you mentioned and it was successful:

Command:

export -c 1111132945210691597 -t ... --media --reuse-media

Also tried in a few other terminals, couldn't reproduce the issue.

It might be that the thread name gets printed because it is successful, and the thread that comes after that one is the one that causes the error. I assumed it was that thread because it is always the same one that gets printed right before the error, but I didn't consider that DCE would always process threads in the same order from run to run.

I'm attempting to get all thread id's from the server and running export channel on each one to see if I can find the thread that fails.

[pinntokuru]

After some diving I've found it is not thread 1111132945210691597 but 1110958658331881614 that is causing the error. It looks like it might be due to the length of the title. This is the error I'm getting:

$TOKEN = "TokenHere"
$SERVERID = "617136488840429598"
$EXPORTFORMAT = "Json"
$EXPORTDIRECTORY = "Z:\Discord-Backups\exports\TMW\%b---00-00-00\"
$MEDIADIR = "Z:\Discord-Backups\exports\TMW\_media\" 

Z:\Discord\dce\DiscordChatExporter.Cli.exe exportguild -t $TOKEN -g $SERVERID -f $EXPORTFORMAT  --after "2023-05-22" -o $EXPORTDIRECTORY --before "2023-05-29" --media --reuse-media --markdown false --include-threads all --include-vc false --media-dir $MEDIADIR --filter ~has:invite *> Z:\Discord\runchannelslogcombined.txt 

Japanese Learning / questions-forum / What does the ? do in  
????????????????????????????????????????
????????????????????????????????: 0%
ERROR
DiscordChatExporter.Core.Exceptions.DiscordChatExporterException: Failed to export message #1110958658331881614 in channel 'What does the ? do in  ????????????????????????????????????????????????????????????????????????' (#1110958658331881614) of guild 'TheMoeWay (#617136488840429598)'.
    System.IO.IOException: The filename, directory name, or volume label syntax is incorrect. : 'Z:\Discord-Backups\exports\TMW\2023-05-29---00-00-00\TheMoeWay - questions-forum - What does the ? do in  ???????????????????????????????????????????????????????????????????????? [1110958658331881614] (2023-05-22 to 2023-05-29).json'.
      at Microsoft.Win32.SafeHandles.SafeFileHandle.Open(String fullPath, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize, Nullable`1 unixCreateMode) 
      at System.IO.Strategies.OSFileStreamStrategy..ctor(String path, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize, Nullable`1 unixCreateMode) 
      at DiscordChatExporter.Core.Exporting.MessageExporter.CreateMessageWriter(String filePath, ExportFormat format, ExportContext context) in D:\a\DiscordChatExporter\DiscordChatExporter\DiscordChatExporter.Core\Exporting\MessageExporter.cs:108
      at DiscordChatExporter.Core.Exporting.MessageExporter.GetWriterAsync(CancellationToken cancellationToken) in D:\a\DiscordChatExporter\DiscordChatExporter\DiscordChatExporter.Core\Exporting\MessageExporter.cs:57
      at DiscordChatExporter.Core.Exporting.MessageExporter.ExportMessageAsync(Message message, CancellationToken cancellationToken) in D:\a\DiscordChatExporter\DiscordChatExporter\DiscordChatExporter.Core\Exporting\MessageExporter.cs:68
      at DiscordChatExporter.Core.Exporting.ChannelExporter.ExportChannelAsync(ExportRequest request, IProgress`1 progress, CancellationToken cancellationToken) in D:\a\DiscordChatExporter\DiscordChatExporter\DiscordChatExporter.Core\Exporting\ChannelExporter.cs:87
  at DiscordChatExporter.Core.Exporting.ChannelExporter.ExportChannelAsync(ExportRequest request, IProgress`1 progress, CancellationToken cancellationToken) in D:\a\DiscordChatExporter\DiscordChatExporter\DiscordChatExporter.Core\Exporting\ChannelExporter.cs:92
  at DiscordChatExporter.Core.Exporting.ChannelExporter.ExportChannelAsync(ExportRequest request, IProgress`1 progress, CancellationToken cancellationToken) in D:\a\DiscordChatExporter\DiscordChatExporter\DiscordChatExporter.Core\Exporting\ChannelExporter.cs:70
  at DiscordChatExporter.Core.Exporting.ChannelExporter.ExportChannelAsync(ExportRequest request, IProgress`1 progress, CancellationToken cancellationToken) in D:\a\DiscordChatExporter\DiscordChatExporter\DiscordChatExporter.Core\Exporting\ChannelExporter.cs:105
  at DiscordChatExporter.Cli.Commands.Base.ExportCommandBase.<>c__DisplayClass63_2.<<ExportAsync>b__2>d.MoveNext() 
  at DiscordChatExporter.Cli.Utils.Extensions.ConsoleExtensions.StartTaskAsync(ProgressContext context, String description, Func`2 performOperationAsync) 
  at DiscordChatExporter.Cli.Commands.Base.ExportCommandBase.<>c__DisplayClass63_1.<<ExportAsync>b__1>d.MoveNext() 
  at System.Threading.Tasks.Parallel.<>c__53`1.<<ForEachAsync>b__53_0>d.MoveNext() 
  at DiscordChatExporter.Cli.Commands.Base.ExportCommandBase.<>c__DisplayClass63_0.<<ExportAsync>b__0>d.MoveNext() 
  at Spectre.Console.Progress.<>c__DisplayClass31_0.<<StartAsync>b__0>d.MoveNext() in \_\src\Spectre.Console\Live\Progress\Progress.cs:103
  at Spectre.Console.Progress.<>c__DisplayClass32_0`1.<<StartAsync>b__0>d.MoveNext() in \_\src\Spectre.Console\Live\Progress\Progress.cs:138
  at Spectre.Console.Internal.DefaultExclusivityMode.RunAsync[T](Func`1 func) in \_\src\Spectre.Console\Internal\DefaultExclusivityMode.cs:40
  at Spectre.Console.Progress.StartAsync[T](Func`2 action) in \_\src\Spectre.Console\Live\Progress\Progress.cs:121
  at Spectre.Console.Progress.StartAsync(Func`2 action) in \_\src\Spectre.Console\Live\Progress\Progress.cs:101
  at DiscordChatExporter.Cli.Commands.Base.ExportCommandBase.ExportAsync(IConsole console, IReadOnlyList`1 channels) 
  at DiscordChatExporter.Cli.Commands.ExportGuildCommand.ExecuteAsync(IConsole console) 
  at CliFx.CliApplication.RunAsync(ApplicationSchema applicationSchema, CommandInput commandInput) in \_\CliFx\CliApplication.cs:163
  at CliFx.CliApplication.RunAsync(IReadOnlyList`1 commandLineArguments, IReadOnlyDictionary`2 environmentVariables) in \_\CliFx\CliApplication.cs:204

This is the actual title of the thread:

However, when running export directly on the channel I get a different error:

Z:\Discord\dce\DiscordChatExporter.Cli.exe export -t "TokenHere" -c "1110958658331881614" -f "Json" --after "2023-05-22" --before "2023-05-29" --media --reuse-media --markdown false --media-dir "Z:\Discord-Backups\exports\TheMoeWay\_media\" --filter ~has:invite -o "Z:\Discord-Backups\exports\TheMoeWay\%b---00-00-00\"

 Unhandled exception. System.ArgumentOutOfRangeException: Index and length must refer to a location within the string. (Parameter 'length')
   at System.String.ThrowSubstringArgumentOutOfRange(Int32 startIndex, Int32 length)
   at System.String.Substring(Int32 startIndex, Int32 length)
   at Spectre.Console.Rendering.Segment.SplitOverflow(Segment segment, Nullable`1 overflow, Int32 maxWidth) in /_/src/Spectre.Console/Rendering/Segment.cs:line 371
   at Spectre.Console.Paragraph.SplitLines(Int32 maxWidth) in /_/src/Spectre.Console/Widgets/Paragraph.cs:line 236
   at Spectre.Console.Paragraph.Render(RenderOptions options, Int32 maxWidth) in /_/src/Spectre.Console/Widgets/Paragraph.cs:line 141
   at Spectre.Console.Rendering.Renderable.Spectre.Console.Rendering.IRenderable.Render(RenderOptions options, Int32 maxWidth) in /_/src/Spectre.Console/Rendering/Renderable.cs:line 19
   at Spectre.Console.Markup.Render(RenderOptions options, Int32 maxWidth) in /_/src/Spectre.Console/Widgets/Markup.cs:line 54
   at Spectre.Console.Rendering.Renderable.Spectre.Console.Rendering.IRenderable.Render(RenderOptions options, Int32 maxWidth) in /_/src/Spectre.Console/Rendering/Renderable.cs:line 19
   at Spectre.Console.TableRenderer.Render(TableRendererContext context, List`1 columnWidths) in /_/src/Spectre.Console/Widgets/Table/TableRenderer.cs:line 31
   at Spectre.Console.Table.Render(RenderOptions options, Int32 maxWidth) in /_/src/Spectre.Console/Widgets/Table/Table.cs:line 147
   at Spectre.Console.Rendering.Renderable.Spectre.Console.Rendering.IRenderable.Render(RenderOptions options, Int32 maxWidth) in /_/src/Spectre.Console/Rendering/Renderable.cs:line 19
   at Spectre.Console.Rendering.JustInTimeRenderable.Render(RenderOptions context, Int32 width) in /_/src/Spectre.Console/Rendering/JustInTimeRenderable.cs:line 21
   at Spectre.Console.Rendering.Renderable.Spectre.Console.Rendering.IRenderable.Render(RenderOptions options, Int32 maxWidth) in /_/src/Spectre.Console/Rendering/Renderable.cs:line 19
   at Spectre.Console.TableRenderer.Render(TableRendererContext context, List`1 columnWidths) in /_/src/Spectre.Console/Widgets/Table/TableRenderer.cs:line 31
   at Spectre.Console.Table.Render(RenderOptions options, Int32 maxWidth) in /_/src/Spectre.Console/Widgets/Table/Table.cs:line 147
   at Spectre.Console.Rendering.Renderable.Spectre.Console.Rendering.IRenderable.Render(RenderOptions options, Int32 maxWidth) in /_/src/Spectre.Console/Rendering/Renderable.cs:line 19
   at Spectre.Console.Rendering.JustInTimeRenderable.Render(RenderOptions context, Int32 width) in /_/src/Spectre.Console/Rendering/JustInTimeRenderable.cs:line 21
   at Spectre.Console.Rendering.Renderable.Spectre.Console.Rendering.IRenderable.Render(RenderOptions options, Int32 maxWidth) in /_/src/Spectre.Console/Rendering/Renderable.cs:line 19
   at Spectre.Console.Padder.Render(RenderOptions options, Int32 maxWidth) in /_/src/Spectre.Console/Widgets/Padder.cs:line 69
   at Spectre.Console.Rendering.Renderable.Spectre.Console.Rendering.IRenderable.Render(RenderOptions options, Int32 maxWidth) in /_/src/Spectre.Console/Rendering/Renderable.cs:line 19
   at Spectre.Console.Rendering.LiveRenderable.Render(RenderOptions options, Int32 maxWidth)+MoveNext() in /_/src/Spectre.Console/Live/LiveRenderable.cs:line 78
   at System.Collections.Generic.List`1.AddRange(IEnumerable`1 collection)
   at Spectre.Console.RenderableExtensions.GetSegments(IAnsiConsole console, RenderOptions options, IEnumerable`1 renderables) in /_/src/Spectre.Console/Extensions/RenderableExtensions.cs:line 37
   at Spectre.Console.RenderableExtensions.GetSegments(IRenderable renderable, IAnsiConsole console) in /_/src/Spectre.Console/Extensions/RenderableExtensions.cs:line 29
   at Spectre.Console.AnsiBuilder.Build(IAnsiConsole console, IRenderable renderable) in /_/src/Spectre.Console/Internal/Backends/Ansi/AnsiBuilder.cs:line 17
   at Spectre.Console.AnsiConsoleBackend.Write(IRenderable renderable) in /_/src/Spectre.Console/Internal/Backends/Ansi/AnsiConsoleBackend.cs:line 30
   at Spectre.Console.AnsiConsoleFacade.Write(IRenderable renderable) in /_/src/Spectre.Console/Internal/Backends/AnsiConsoleFacade.cs:line 40
   at Spectre.Console.ProgressContext.Refresh() in /_/src/Spectre.Console/Live/Progress/ProgressContext.cs:line 178
   at Spectre.Console.ProgressRefreshThread.Run() in /_/src/Spectre.Console/Live/Progress/ProgressRefreshThread.cs:line 45
   at System.Threading.Thread.StartCallback()

[Tyrrrz]

The first error is definitely due to the file path length. You can configure your system to support longer file paths to avoid this issue: https://learn.microsoft.com/en-us/windows/win32/fileio/maximum-file-path-limitation?tabs=registry#enable-long-paths-in-windows-10-version-1607-and-later

As for the second error, I was able to reproduce it myself with the command line you've provided. I'll try to debug.

[Tyrrrz]

Looks like the issue comes from the string 神様達が下界に来る前は、魔法は特定の種族の専売特許に過ぎなかった。けれど、神様達の『恩恵』はいかなる者でも魔法を発現させることを可能としたのだ。, which the Spectre.Console library fails to split into segments correctly, as it overflows the console's max width.

I've reported the error in its corresponding repository: spectreconsole/spectre.console#1787