Is it possible to use this project to debug userland processes running inside the Linux kernel guest from the QEMU GDB stub, and without Xen?

[cirosantilli]

Salut Mathieu,

I've been looking for such capability for a while, and this project seems really close.

I summarized my findings so far: https://stackoverflow.com/questions/9561546/thread-aware-gdb-for-the-linux-kernel/51515470#51515470

My main question is: is that possible with this project? The main thing which might not work is that I'm not using Xen, just the QEMU GDB stub directly.

But you seem to be parsing Linux kernel process data structures, which is the hard part.

Or do you know any other project which has achieved this?

[Wenzel]

Hi,

I've been looking for such capability for a while, and this project seems really close.

You are looking for a guest-aware GDB stub.
Yes, you are in the right place.

My main question is: is that possible with this project?

It should be.
I already implemented the necessary guest awareness for Windows (XP).

I looked at Linux internals, got interested in the scheduler, but couldn't find the resources to have the whole picture.
I stopped at a function in core/sched.c:switch_to() which is scheduling a new task.
That's where I wanted to trap to intercept tasks and find the targeted process.

At the moment the implementation is silly, you can follow how I'm attaching to an existing process for Linux here:
https://github.com/Wenzel/pyvmidbg/blob/master/vmidbg/linuxdebugcontext.py#L54

I try to find the process descriptor, find the CR3, and wait for the CR3 to be loaded, and then pause the VM.
As a consequence I'm pausing the vm somewhere in the middle of the scheduler, when it is scheduling a new task...

That's far from ideal.

If you are ready to look at Linux internals, feel free to implement it.

But you seem to be parsing Linux kernel process data structures, which is the hard part.

I'm parsing Rekall profiles, which are JSON file containing structs, constants, symbols, etc.
Yes it's the hard part.

Look how i'm using these profiles to read the Windows process and thread descriptor fields:
https://github.com/Wenzel/pyvmidbg/blob/master/vmidbg/windowsdebugcontext.py#L33

The main thing which might not work is that I'm not using Xen, just the QEMU GDB stub directly.

Xen is the only hypervisor fully supported by Libvmi at the moment.
the libvmi driver has memory, VCPU and event access.

I created a project to help bring vmi on KVM, and an appropriate libvmi driver:
https://github.com/KVM-VMI/kvm-vmi
Also feel free to join the effort.

I will update the README to state what hypervisor is supported, as it brings confusion.

Or do you know any other project which has achieved this?

I know people working on a KD stub inspecting Windows OS, but not Linux.

Apart from the solution you listed in your SO post, that's it.

You might want to look at a presentation I did on the topic:
https://drive.google.com/file/d/1ZMUszfwWDOljdDfPOJgkEfSabNy0UAJR/view

Thanks !

[Wenzel]

I just updated the README, adding a section about the state of VMI support in hypervisors today.

[cirosantilli]

Awesome, thanks for this info! Extending this library does seem like the best way to achieve the solution I've seen so far!

I can't work on it now, but I'll update the answer to mention this, and ping the people who were working on an in-GDB solution.

Feel free to close this ticket, or keep it open: likely this should be split into two: no-Xen support and Linux support.

[Wenzel]

@pwnosaur Update: pyvmidbg now supports KVM (non-Xen)
related: #40

[cirosantilli]

O M G! Any reason why it should not work on QEMU BTW? If it might work I'll try it out.

[Wenzel]

It depends what you mean by QEMU:

• full virtualization (QEMU/KVM) -> yes ✔️
• binary translation (QEMU only) -> no 🔴

[cirosantilli]

OK, was mostly thinking about binary translation :-( I have to understand better how this project works at some point and why it works for one and not the other. Fantastic news anyways!

[pwnosaur]

@cirosantilli Hey, yes we are parsing the Linux kernel structures, and injecting actual breakpoints, but there are a couple of things I need to make sure they are stable first and test then I will be updating and submitting it to the repository soon enough as I've been inactive lately.