<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">On 29 March 2015 at 08:14, Elazar Leibovich <span dir="ltr"><<a href="mailto:elazarl@gmail.com" target="_blank">elazarl@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div>4) I really think nowadays text is a bit obsolete. I can hardly think of a case where text would be more convenient than, say, json or HTML. Web browser is at your fingertips, and HTML table is easier to handle than whitespace separated output based on your terminal size. <tr><td>eth0</td><td>100</td></tr> is just as grep'able as the tab separated version, and is easier to view, sort, etc.</div></div></div></blockquote><div><br></div><div>By "text" I DID mean JSON. I think that using JSON should address all the concerns above and keep the protocol future-proof. I didn't find kernel-specific JSON parser implementations but suspect that it should be possible to use any simple user-space C implementation.</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div><br></div><div>[0] <a href="https://github.com/elazarl/cpu_affinity/blob/master/tracecpu.d" target="_blank">https://github.com/elazarl/cpu_affinity/blob/master/tracecpu.d</a></div><div>[1] <a href="https://github.com/elazarl/cpu_affinity/blob/master/test/linux/plot_ftrace_sched_switch.py" target="_blank">https://github.com/elazarl/cpu_affinity/blob/master/test/linux/plot_ftrace_sched_switch.py</a> </div></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Sat, Mar 28, 2015 at 12:49 AM, Amos Shapira <span dir="ltr"><<a href="mailto:amos.shapira@gmail.com" target="_blank">amos.shapira@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">If serialisation (aka "marshalling") is considered, how about making it text based?<div>Then you can use simple shell tools to talk to it.<div><br></div></div></div><div class="gmail_extra"><div><div><br><div class="gmail_quote">On 27 March 2015 at 22:34, Elazar Leibovich <span dir="ltr"><<a href="mailto:elazarl@gmail.com" target="_blank">elazarl@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">IMHO, C structs are no way near as usable as proper serialization<br>
format. For example, what about optional fields? What about variable<br>
length array? What about binary backwards compatibility? What about<br>
supporting other languages? It's not trivial to take a C struct and<br>
generate the proper struct.unpack string for it.<br>
<br>
Look at the complexity in perf_event_open(2), just parsing the event<br>
stream takes a good chunk of code[0], with many potential bugs.<br>
Parsing it with protobuf (or one of the other serialization formats)<br>
would take three lines or so, would be more efficient, and would be<br>
easier to program against, and less prone to bugs, etc.<br>
<br>
[0] Here is my take, and it's not even complete<br>
<a href="https://gist.github.com/elazarl/c8404686e71ef0b36cc7" target="_blank">https://gist.github.com/elazarl/c8404686e71ef0b36cc7</a><br>
<br>
On Fri, Mar 27, 2015 at 12:26 PM, guy keren <<a href="mailto:guy.choo.keren@gmail.com" target="_blank">guy.choo.keren@gmail.com</a>> wrote:<br>
><br>
> i imagine, if you use the proper 'packing' pragmas, you can simply mempcy<br>
> structures, without really writing serialization code (there's no endianess<br>
> issues, with both sides running on the same host, by definition).<br>
><br>
> --guy<br>
><br>
><br>
> On 03/27/2015 10:03 AM, Elazar Leibovich wrote:<br>
>><br>
>> Thanks, didn't know netlink.<br>
>><br>
>> You still need a solution to parse the sent message, where protocol<br>
>> buffers etc, can help. (e.g., binary data into struct<br>
>> mymodule_request).<br>
>><br>
>> Or am I missing something?<br>
>><br>
>> On Fri, Mar 27, 2015 at 3:33 AM, guy keren <<a href="mailto:guy.choo.keren@gmail.com" target="_blank">guy.choo.keren@gmail.com</a>><br>
>> wrote:<br>
>>><br>
>>><br>
>>> take a look at this:<br>
>>><br>
>>><br>
>>> <a href="http://www.linuxfoundation.org/collaborate/workgroups/networking/generic_netlink_howto" target="_blank">http://www.linuxfoundation.org/collaborate/workgroups/networking/generic_netlink_howto</a><br>
>>><br>
>>> (link got broken - place it all on a single line)<br>
>>><br>
>>> --guy<br>
>>><br>
>>><br>
>>> On 03/26/2015 11:36 PM, Elazar Leibovich wrote:<br>
>>>><br>
>>>><br>
>>>> Hi,<br>
>>>><br>
>>>> I'm writing a kernel module, and I want to expose some debug<br>
>>>> information about it.<br>
>>>><br>
>>>> The debug information is often of the form of request-response.<br>
>>>><br>
>>>> For example:<br>
>>>><br>
>>>> - Hey module, what's up with data at 0xffffe8ff0040c000?<br>
>>>> - Cached, populated two hours ago.<br>
>>>><br>
>>>> - Hey module, please invalidate data at 0xffffe8ff0002cb00<br>
>>>> - Sure thing.<br>
>>>><br>
>>>> - Hey module, please record all accesses to 0xffffe8ff0006bbf0.<br>
>>>> - OK, ask me again for stats-5<br>
>>>> ...<br>
>>>> - Hey module, what's in stats-5?<br>
>>>> - So far, 41 accesses by 22 users.<br>
>>>><br>
>>>> Now, the question is, what is a good design to expose this information.<br>
>>>><br>
>>>> I think that the most reasonable way to interact with userspace is<br>
>>>> through a debugfs file.<br>
>>>><br>
>>>> The user would open the debugfs file in read+write mode, would write a<br>
>>>> request, and accept a response from it.<br>
>>>><br>
>>>> As I see it, there are two fundamental problems needs to be solved:<br>
>>>><br>
>>>> - Parsing the request from the client.<br>
>>>> - Writing the response in a recognizeable format.<br>
>>>><br>
>>>> A simple solution I first came up with, is to use a ad-hoc<br>
>>>> request-response format. In my case, request and response are line<br>
>>>> delimited, request is a hex address, and response is a translated hex<br>
>>>> address.<br>
>>>><br>
>>>> Here is the relevant snippet.<br>
>>>><br>
>>>> struct pipe {<br>
>>>> DECLARE_KFIFO(fifo, T, (1<<4));<br>
>>>> wait_queue_head_t queue;<br>
>>>> char buf[100];<br>
>>>> int buflen;<br>
>>>> char resp[100];<br>
>>>> int resp_len;<br>
>>>> };<br>
>>>> static DEFINE_MUTEX(mutex);<br>
>>>> static int open(struct inode *inode, struct file *file)<br>
>>>> {<br>
>>>> struct pipe *pipe;<br>
>>>> if (!(file->f_mode & FMODE_READ) || !(file->f_mode & FMODE_READ))<br>
>>>> {<br>
>>>> pr_warn("must open with O_RDWR\n");<br>
>>>> return -EINVAL;<br>
>>>> }<br>
>>>> mutex_lock(&mutex);<br>
>>>> pipe = kzalloc(sizeof(*pipe), GFP_KERNEL);<br>
>>>> INIT_KFIFO(pipe->fifo);<br>
>>>> init_waitqueue_head(&pipe->queue);<br>
>>>> file->private = pipe;<br>
>>>> }<br>
>>>><br>
>>>> static int write(struct file *file, const char __user *ubuf, size_t<br>
>>>> count, loff_t *ppos)<br>
>>>> {<br>
>>>> char *eol;<br>
>>>> size_t n = min_t(size_t, count, sizeof(pipe->buf));<br>
>>>> struct pipe *pipe = file->private_data;<br>
>>>> if (copy_from_user(&pipe->buf[pipe->buflen], ubuf, n)<br>
>>>> return -EFAULT;<br>
>>>> eol = memchr(buf, '\n', n);<br>
>>>> if (eol == NULL)<br>
>>>> return count;<br>
>>>> *eol = '\0';<br>
>>>> // TODO: wait when queue full<br>
>>>> if (!kfifo_in(&pipe->fifo, processLine(buf), 1)<br>
>>>> return -EFAULT;<br>
>>>> wake_up_interruptible(&pipe->queue);<br>
>>>> memmove(&pipe->buf[0], &pipe->buf[n], pipe->buflen-n);<br>
>>>> }<br>
>>>><br>
>>>> static int read(struct file *file, const char __user *ubuf, size_t<br>
>>>> count, loff_t *ppos)<br>
>>>> {<br>
>>>> struct pipe *pipe = file->private_data;<br>
>>>> T req;<br>
>>>> wait_event_interruptible(pipe->queue, kfifo_out(&pipe->fifo, &req,<br>
>>>> 1));<br>
>>>> process_request(req, &pipe->resp, &pipe->resp_len);<br>
>>>> if (count < pipe->resp_len)<br>
>>>> return -EFAULT; // TODO: handle copy to client in parts<br>
>>>> if (copy_to_user(userbuf, buf, pipe->resp_len))<br>
>>>> return -EFAULT;<br>
>>>> }<br>
>>>><br>
>>>> Usage is:<br>
>>>><br>
>>>> fd = io.FileIO("/debug/mymodule/file", "r+")<br>
>>>> fd.write('req...')<br>
>>>> print fd.read(100)<br>
>>>><br>
>>>> This is not so robust, for many reasons (look how many bugs are in<br>
>>>> this small and simple snippet), and some parts need to be repeated for<br>
>>>> each input type.<br>
>>>><br>
>>>> What I've had in mind, in similar fashion to <a href="http://grpc.io" target="_blank">grpc.io</a>, have the user<br>
>>>> write a size prefixed protocol buffer object to the file, and<br>
>>>> similarly read it as a response.<br>
>>>><br>
>>>> Something like:<br>
>>>><br>
>>>> fd = io.FileIO("/debug/mymodule/file", "r+")<br>
>>>> fd.write(myReq.SerializeToString())<br>
>>>> len = struct.unpack("<i", fd.read(4))<br>
>>>> Resp.ParseFromString(fd.read(len))<br>
>>>><br>
>>>> I believe it is not hard to create a kernel compatible protocol buffer<br>
>>>> code generator.<br>
>>>><br>
>>>> When you have this in place, you have to write a very simple logic to<br>
>>>> add a new functionality to the debugfs file. Handler would essentially<br>
>>>> get pointers to a request struct, and a response struct, and would<br>
>>>> need to fill out the response struct.<br>
>>>><br>
>>>> Are there similar solutions?<br>
>>>> What problems might my approach cause?<br>
>>>> Is there a better idea for this problem altogether?<br>
>>>><br>
>>>> Thanks,<br>
>>>><br>
>>>> _______________________________________________<br>
>>>> Linux-il mailing list<br>
>>>> <a href="mailto:Linux-il@cs.huji.ac.il" target="_blank">Linux-il@cs.huji.ac.il</a><br>
>>>> <a href="http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il" target="_blank">http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il</a><br>
>>>><br>
>>><br>
>>><br>
>>> _______________________________________________<br>
>>> Linux-il mailing list<br>
>>> <a href="mailto:Linux-il@cs.huji.ac.il" target="_blank">Linux-il@cs.huji.ac.il</a><br>
>>> <a href="http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il" target="_blank">http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il</a><br>
><br>
><br>
<br>
_______________________________________________<br>
Linux-il mailing list<br>
<a href="mailto:Linux-il@cs.huji.ac.il" target="_blank">Linux-il@cs.huji.ac.il</a><br>
<a href="http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il" target="_blank">http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il</a><br>
</blockquote></div><br><br clear="all"><span class="HOEnZb"><font color="#888888"><div><br></div></font></span></div></div><span class="HOEnZb"><font color="#888888"><span><font color="#888888">-- <br><div><div dir="ltr"><a href="http://au.linkedin.com/in/gliderflyer" target="_blank"><img src="https://static.licdn.com/scds/common/u/img/webpromo/btn_viewmy_160x25.png"></a><br></div></div>
</font></span></font></span></div>
</blockquote></div><br></div>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature"><div dir="ltr"><a href="http://au.linkedin.com/in/gliderflyer" target="_blank"><img src="https://static.licdn.com/scds/common/u/img/webpromo/btn_viewmy_160x25.png"></a><br></div></div>
</div></div>