SMOLNET PORTAL home about changes
[root][parent][raw file]
blog.netbsd.org.atom.xml - sfeed_tests - sfeed tests and RSS and Atom files
(URL) git clone git://git.codemadness.org/sfeed_tests (git://git.codemadness.org)
(DIR) Log
(DIR) Files
(DIR) Refs
(DIR) README
(DIR) LICENSE
---
blog.netbsd.org.atom.xml (280500B)
---
1 <?xml version="1.0" encoding='utf-8'?>
2 <?xml-stylesheet type="text/xsl" href="https://blog.netbsd.org/roller-ui/styles/atom.xsl" media="screen"?><feed xmlns="http://www.w3.org/2005/Atom">
3 <title type="html">NetBSD Blog</title>
4 <subtitle type="html"></subtitle>
5 <id>https://blog.netbsd.org/tnf/feed/entries/atom</id>
6 <link rel="self" type="application/atom+xml" href="https://blog.netbsd.org/tnf/feed/entries/atom" />
7 <link rel="alternate" type="text/html" href="https://blog.netbsd.org/tnf/" />
8 <updated>2022-03-09T23:57:54+00:00</updated>
9 <generator uri="http://roller.apache.org" version="5.0.3 (1388864191739:dave)">Apache Roller (incubating)</generator>
10 <entry>
11 <id>https://blog.netbsd.org/tnf/entry/making_rockpro64_a_netbsd_server</id>
12 <title type="html">Making RockPro64 a NetBSD Server</title>
13 <author><name>matthew green</name></author>
14 <link rel="alternate" type="text/html" href="https://blog.netbsd.org/tnf/entry/making_rockpro64_a_netbsd_server"/>
15 <published>2022-03-09T23:57:54+00:00</published>
16 <updated>2022-03-09T23:57:54+00:00</updated>
17 <category term="/General" label="General" />
18 <category term="rockpro64" scheme="http://roller.apache.org/ns/tags/" />
19 <category term="aarch64" scheme="http://roller.apache.org/ns/tags/" />
20 <category term="pine64" scheme="http://roller.apache.org/ns/tags/" />
21 <category term="evbarm" scheme="http://roller.apache.org/ns/tags/" />
22 <category term="aarch64eb" scheme="http://roller.apache.org/ns/tags/" />
23 <content type="html">&lt;p&gt;The time has come to upgrade my SunBlade 2500s to something more power friendly and faster. I&apos;d already removed one CPU and thus half the ram from two of these systems to reduce their power consumption, but it&apos;s still much higher than it could be.&lt;/p&gt;
24
25 &lt;p&gt;After much searching, I&apos;ve decided on &lt;a href=&quot;https://wiki.pine64.org/wiki/ROCKPro64&quot;&gt;Pine64&apos;s RockPro64&lt;/a&gt; 4GiB ram model (technically, only 3.875GiB ram.) Pine64 make SBCs, laptops, phones, and various other mostly ARM gadgets, and the RockPro64 has the fastest CPU they ship (Rockchip RK3399), and can use a small &quot;NAS Case&quot;, that is sufficient to house 2 HDDs and, at a stretch, upto 6 SSDs (cooling would become quite an issue at this point.)&lt;/p&gt;
26
27 &lt;p&gt;In my SATA setup, I have 3 SSDs with a JMicron 585 card in the PCIe slot, two SSDs in the NAS case SSD region, and the third is in the HDD region with an adapter. I have used two SATA power splitters to convert the NAS case&apos;s 2 SATA power ports into 4, with the 4th one also powering a Noctua case fan. The cabling is not great with this, with enough SATA power cabling for 6 devices to lay. Probably a bespoke power cable to connect to the RockPro64 would make this nicer and probably improve cooling slightly, but I&apos;m just using off-the-shelf components for now.&lt;/p&gt;
28
29
30 &lt;p&gt;In the last year or so I&apos;ve been working on making NetBSD/arm64 big-endian more featureful. In particular, I&apos;ve added support for:
31 &lt;ul&gt;&lt;li&gt;other-endian access disklabels, FFS file-systems in the NetBSD libsa&lt;/li&gt;
32 &lt;li&gt;the &quot;look 64 sectors later&quot; for RAIDFrame partitions in MI efiboot (the x86 specific efiboot has more extensive support for this that should be ported over)&lt;/li&gt;
33 &lt;li&gt;other-endian access to RAIDFrame labels in the kernel&lt;/li&gt;
34 &lt;li&gt;updated the U-Boot package and featureset to include AHCI/SATA support, workaround some bugs and fix the newer U-Boot SPI loader location, and figured out how to default loading from either SATA or NVMe&lt;/li&gt;
35 &lt;/ul&gt;&lt;/p&gt;
36 &lt;p&gt;There are not too many special actions needed for this sort of setup compared to a normal NetBSD or Arm system. While I built my installations by hand, the standard NetBSD Arm images are suitable for this task. It&apos;s easiest to start from an SD card with the RockPro64 u-boot installed. There are two U-Boot images available, one for SD/eMMC, and one for SPI (there is an odd problem with the early SPI loader that requires a portion of the image to be different.) The pkgsrc package for sysutils/u-boot-rockpro64 version 2022.01 has these suggested methods for installing the U-Boot image (this package should be buildable on any supported pkgsrc platform).&lt;/p&gt;
37
38 &lt;p&gt;To install U-Boot into the SD/eMMC card (can run on any system, the image must be written at 32KiB into the device):
39 &lt;pre&gt;
40 # dd if=/usr/pkg/share/u-boot/rockpro64/rksd_loader.img seek=64 of=/dev/rld0c
41 &lt;/pre&gt;
42 &lt;p&gt;
43 To install U-Boot into the SPI flash (must be run on the host, and lives at the very start of the device:
44 &lt;pre&gt;
45 # dd if=/usr/pkg/share/u-boot/rockpro64/rkspi_loader.img bs=64k of=/dev/spiflash0
46 &lt;/pre&gt;
47 &lt;/p&gt;
48
49 &lt;p&gt;When booting from NVMe or SATA, one must drop to the U-Boot prompt and adjust the &quot;boot_targets&quot; value to put scsi* (SATA) or nvme* ahead of the mmc* and usb* options.&lt;/p&gt;
50
51 &lt;p&gt;The original value for me:
52 &lt;pre&gt;
53 =&gt; printenv boot_targets
54 boot_targets=mmc1 usb0 mmc0 nvme0 scsi0 pxe dhcp sf0
55 &lt;/pre&gt;
56 &lt;/p&gt;
57
58 &lt;p&gt;Which is then adjusted and saved with eg:
59 &lt;pre&gt;
60 =&gt; setenv boot_targets nvme0 scsi0 mmc1 usb0 mmc0 pxe dhcp sf0
61 =&gt; saveenv
62 Saving Environment to SPIFlash... Erasing SPI flash...Writing to SPI flash...done
63 OK
64 &lt;/pre&gt;&lt;/p&gt;
65
66 &lt;p&gt;(In this list, mmc1 is the SD slot, mmc0 is the eMMC card, pxe and dhcp are netbooting, and sf0 attempts to load further U-Boot scripts from SPI.)&lt;/p&gt;
67
68 &lt;p&gt;There are some minor issues with the RockPro64. It has no ability to use ECC memory. It only comes with 1 PCIe 4x slot, and Rockchip errata limited this to PCIe 1.x (though no NetBSD users encounted any issues with PCIe 2.x enabled, and this can be forced back on via a DTS patch.) It is possible to use a PCIe bridge (I have never done this, though I would like to try in the future) to enable more devices for booting, or to enable both a network and storage device.&lt;/p&gt;
69 </content>
70 </entry>
71 <entry>
72 <id>https://blog.netbsd.org/tnf/entry/project_report_add_support_for</id>
73 <title type="html">Project Report: Add support for chdir(2) support in posix_spawn(3)</title>
74 <author><name>martin</name></author>
75 <link rel="alternate" type="text/html" href="https://blog.netbsd.org/tnf/entry/project_report_add_support_for"/>
76 <published>2021-11-22T18:01:54+00:00</published>
77 <updated>2021-11-23T16:38:20+00:00</updated>
78 <category term="/Development" label="Development" />
79 <category term="projects" scheme="http://roller.apache.org/ns/tags/" />
80 <category term="tnf" scheme="http://roller.apache.org/ns/tags/" />
81 <category term="gsoc" scheme="http://roller.apache.org/ns/tags/" />
82 <summary type="html">&lt;p&gt;Piyush Sachdeva finished the &quot;add chdir support to posix_spawn(3)&quot; project and reports about his work and experience.
83 His code is already in -current and will be part of NetBSD 10.&lt;/p&gt;
84 &lt;p&gt;Originally submitted as a proposal for GSoC, but unfortunately (due to low slot allocations) this project was not part of GSoC.&lt;/p&gt;
85 &lt;p&gt;The NetBSD Foundation decided to nevertheless run the project and funded it.&lt;/p&gt;</summary>
86 <content type="html">&lt;h3&gt;This post was written by Piyush Sachdeva:&lt;/h3&gt;
87
88 &lt;h2&gt;Abstract&lt;/h2&gt;
89
90 &lt;p&gt;The primary goal of the project was to extend posix_spawn(3) to include chdir(2)
91 for the newly created child process. Two functions were supposed to be implemented,
92 namely posix_spawn_file_actions_addchdir() and
93 posix_spawn_file_actions_addfchdir(), to support both chdir(2) and
94 fchdir(2) respectively.
95 posix_spawn() is a POSIX standard method
96 responsible for creating and executing new child processes.&lt;/p&gt;
97
98 &lt;h2&gt;Implementation&lt;/h2&gt;
99 &lt;p&gt;The original code can be found at &lt;a href=&quot;https://github.com/cosmologistPiyush/posix_spawn-chdir/tree/trunk&quot;&gt;my github tree&lt;/a&gt;.&lt;/p&gt;
100 &lt;p&gt;The implementation plan was discussed and made with the guidance of both my
101 mentors Martin Husemann and Joerg Sonnenberger. The plan was divided into
102 three phases each corresponding to the specific part of The NetBSD code-base which is
103 supposed to be touched:&lt;/p&gt;
104
105 &lt;h3&gt;User-Land&lt;/h3&gt;
106 &lt;p&gt;The following actions were performed in the user-land to set things up for the
107 kernel-space.
108 &lt;ul&gt;
109 &lt;li&gt;Add another member to the posix_spawn_file_actions_t struct i.e a union,
110 which would hold the path to chdir to.&lt;/li&gt;
111 &lt;/li&gt;
112 &lt;li&gt;Implement the two functions posix_spawn_file_actions_addchdir()
113 and posix_spawn_file_actions_addfchdir(). These functions would:
114 &lt;ol&gt;
115 &lt;li&gt;allocate memory for another posix_spawn_file_actions_t object in the
116 posix_spawn_file_actions_t array.&lt;/li&gt;
117 &lt;li&gt;take the path/file descriptor from the user as an argument and make the
118 relative field of the newly allocated file actions object, point to it.&lt;/li&gt;
119 &lt;/ol&gt;
120 &lt;/li&gt;
121 &lt;li&gt;The final step was to add the prototypes for the two new functions to the
122 `src/include/spawn.h&apos; header file
123 &lt;/li&gt;
124 &lt;/ul&gt;
125
126 &lt;p&gt;
127 Once the aforementioned changes were made, the only thing left to do was to make the
128 kernel support these two new functions.&lt;/p&gt;
129
130 &lt;h3&gt;Kernel-Space&lt;/h3&gt;
131 &lt;p&gt;The following actions were performed inside the kernel space.&lt;/p&gt;
132 &lt;ul&gt;
133 &lt;li&gt;The three functions in the `src/sys/kern_exec.c&apos; file which correspond to the
134 posix_spawn_file_actions were edited:
135 &lt;ul&gt;
136 &lt;li&gt;posix_spawn_fa_alloc() was adjusted to make sure that the path passed to
137 posix_spawn_file_actions_addchdir() gets copied from the user-land
138 to the kernel-space.
139 &lt;/li&gt;
140 &lt;li&gt;Similarly posix_spawn_fa_free() was adjusted to make sure that the
141 memory allocated in case of FAE_CHDIR gets freed as well.
142 &lt;/li&gt;
143 &lt;li&gt;Finally, two new cases FAE_CHDIR &amp;amp; FAE_FCHDIR were added in the
144 handle_posix_spawn_file_actions(). In each of the cases,
145 a call to one of the two newly created functions (discussed in the next point)
146 do_sys_chdir() and do_sys_fchdir() was made respectively.
147 &lt;/li&gt;
148 &lt;/ul&gt;
149
150 &lt;p&gt;Note: At the time of code integration, a helper function was written by
151 Christos Zoulas. This function aimed to reduce the amount of repeated code
152 in both posix_spawn_fa__free() and posix_spawn_fa_alloc()&lt;/p&gt;
153 &lt;/li&gt;
154
155 &lt;li&gt;Two new functions, similar to the already present sys_chdir() and sys_fchdir() in
156 `src/sys/vfs_syscalls.c&apos; were created. Namely do_sys_chdir() and do_sys_chdir
157 were written with two specific thoughts in mind:
158 &lt;ul&gt;
159 &lt;li&gt;By default sys_chdir() and sys_fchdir() took syscallargs as a parameter.
160 The purpose of the new functions was to replace this with
161 const char * and an int type parameter respectively.&lt;/li&gt;
162 &lt;li&gt;The do_sys_chdir() also replaced UIO_USERSPACE with
163 UIO_SYSSPACE. This was done because the chdir path passed to this
164 function already resided in the Kernel-space due to the change made in
165 posix_spawn_fa_alloc().&lt;/li&gt;
166 &lt;/ul&gt;&lt;/li&gt;
167 &lt;li&gt;Finally, the prototypes for the newly written functions were added to the
168 `src/sys/sys/vfs_syscalls.h&apos; file and this file was also included in the
169 &apos;sys/kern/kern_exec.c&apos;.&lt;/li&gt;
170
171 &lt;/ul&gt;
172
173 &lt;p&gt;Note: Similar to the above changes of user-land and kernel-space, a few tweaks were also made to
174 `src/sys/compat/netbsd/netbsd32.h&apos; and `netbsd32_execve.c&apos;. This was required to help COMPAT_NETBSD32
175 deal with the new file actions member. However, these changes were made at the time of integration
176 by Martin Husemann.&lt;/p&gt;
177
178
179 &lt;p&gt;With most of addition of new features being done, all that remained was testing and documentation.&lt;/p&gt;
180
181
182 &lt;h3&gt;Testing &amp;amp; Documentation&lt;/h3&gt;
183 &lt;ul&gt;
184 &lt;li&gt;A total of ten new test cases have been added to the
185 `src/tests/lib/libc/gen/posix_spawn/t_spawn.c&apos; file.&lt;/li&gt;
186 &lt;li&gt;Three utility functions were also used to aid in testing. Out of the three,
187 one new function was written and two existing functions (filesize()
188 and empty_outfile()) from `t_fileactions.c&apos; were used.
189 To make sure that the 2 existing functions were shared between both the files i.e `t_spawn.c&apos;
190 and `t_fileactions.c&apos; a new header and C file was created, namely `fa_spawn_utils.h&apos; and
191 `fa_spawn_utils.c&apos;.
192 Following this, the bodies of both the functions were moved from
193 `t_fileactions.c&apos; to `fa_spawn_utils.c&apos; and their prototypes were added to the
194 corresponding header file.&lt;/li&gt;
195 &lt;li&gt;The general approach that was taken to all test cases was to make
196 posix_spawn() execute ``/bin/pwd&apos;&apos; and write the output to a file.
197 Then read the file and do string comparison. The third function i.e. check_succes()
198 was written for just this purpose.&lt;/li&gt;
199 &lt;li&gt;The ten test cases cover the following scenarios:
200 &lt;ul&gt;
201 &lt;li&gt;Absolute path test - for both chdir and fchdir.&lt;/li&gt;
202 &lt;li&gt;Relative path test - for both chdir and fchdir.&lt;/li&gt;
203 &lt;li&gt;Trying to open a file instead of directory - for both chdir and fchdir.&lt;/li&gt;
204 &lt;li&gt;Invalid path/file descriptor (fd=-1) - for both chdir and fchdir.&lt;/li&gt;
205 &lt;li&gt;Trying to open a directory without access permissions for chdir.&lt;/li&gt;
206 &lt;li&gt;Opening a closed file descriptor for fchdir.&lt;/li&gt;
207 &lt;/ul&gt;&lt;/li&gt;
208 &lt;li&gt;The first 8 test cases had a lot of repetitive code. Therefore, at the time of integration,
209 another function was created i.e spawn_chdir().
210 This function included a huge chunk of the common code and it did all the heavy lifting
211 for those first 8 test cases.&lt;/li&gt;
212 &lt;/ul&gt;
213
214 &lt;h4&gt;Documentation:&lt;/h4&gt;
215 &lt;p&gt;In this matter, a complete man page is written which explains both
216 posix_spawn_file_actions_addchdir() and posix_spawn_file_actions_addfchdir() in great detail.
217 The content of the manual page is taken from the POSIX documentation provided to us by Robert Elz.
218 &lt;/p&gt;
219
220
221 &lt;h2&gt;Issues&lt;/h2&gt;
222 &lt;p&gt;Since the project was well planned from the beginning, it resulted in few issues.&lt;/p&gt;
223 &lt;ul&gt;
224 &lt;li&gt;The user-land was the most straight forward part of the project and I had no
225 trouble sailing through it.&lt;/li&gt;
226 &lt;li&gt;Kernel space was where things got a bit complicated, as I had to add functionality
227 to pre-existing functions.&lt;/li&gt;
228 &lt;li&gt;I was completely new to using atf(7) and groff(1). Therefore, it took me some time
229 to understand the respective man pages and become comfortable with testing and
230 documentation part.&lt;/li&gt;
231 &lt;/ul&gt;
232
233 &lt;p&gt;
234 Most of the issues faced were generally logistical. As it was my first time doing a
235 kernel project, I was new to building from source, Virtual Machines and other things like SSH.
236 But luckily, I had great help from my mentors and the entire NetBSD community.&lt;/p&gt;
237
238 &lt;h2&gt;Thanks&lt;/h2&gt;
239 &lt;p&gt;I would like to express my heartfelt gratitude to The NetBSD Foundation for
240 giving me this opportunity and sponsoring the Project.
241 This project would not have been possible without the constant support and
242 encouragement of both my mentors Martin Husemann and Joerg Sonnenberger.
243 My gratitude to Christos Zoulas who worked on the crucial part of integrating the code.
244 A special mention to all of the other esteemed NetBSD developers,
245 who have helped me navigate through the thick and thin of this project and
246 have answered even my most trivial questions.&lt;/p&gt;
247 </content>
248 </entry>
249 <entry>
250 <id>https://blog.netbsd.org/tnf/entry/wifi_project_status_update</id>
251 <title type="html">wifi project status update</title>
252 <author><name>martin</name></author>
253 <link rel="alternate" type="text/html" href="https://blog.netbsd.org/tnf/entry/wifi_project_status_update"/>
254 <published>2021-08-26T17:46:53+00:00</published>
255 <updated>2021-08-26T17:46:53+00:00</updated>
256 <category term="/Development" label="Development" />
257 <category term="funded" scheme="http://roller.apache.org/ns/tags/" />
258 <category term="wifi" scheme="http://roller.apache.org/ns/tags/" />
259 <category term="freebsd" scheme="http://roller.apache.org/ns/tags/" />
260 <summary type="html">&lt;p&gt;About a year ago the &lt;a href=&quot;/tnf/entry/wifi_renewal_restarted&quot;&gt;wifi renewal project&lt;/a&gt; got restarted. A lot things happened, but the high hopes of a quick breakthrough and fast merge to mainline did not come true.&lt;/p&gt;
261 &lt;p&gt;Here is where we are today, what needs to be done and how things are planned to move on...&lt;/p&gt;</summary>
262 <content type="html">&lt;p&gt;After initial work on the &lt;a href=&quot;https://wiki.NetBSD.org/Wifi_renewal_on_hg/&quot;&gt;wifi renewal branch&lt;/a&gt; went quite fast and smooth, things have slowed down a bit in the last few months.&lt;/p&gt;
263
264 &lt;p&gt;Most of the slow down was due to me not being available for this type of work for unexpectedly long times - a problem that should be fixed now.&lt;/p&gt;
265
266 &lt;p&gt;However, there were other obstacles and unexpected issues on the way:&lt;/p&gt;
267 &lt;ul&gt;
268 &lt;li&gt;bpf taps are handled differently in the new stack and some slightly obscure site conditions of this had been overlooked in the initial conversion. To make everything work, changes to our bpf framework were needed (and have landed in -current some time ago now).&lt;/li&gt;
269 &lt;li&gt;Many wifi drivers seem to be in a, let&apos;s say, slightly fragile state. When testing the random collection of wifi hardware that I acquired during this project in -current, many drivers did not work at first try and often I was able to provoke kernel panics quickly.
270 This is not a happy base to start converting drivers from.&lt;/li&gt;
271 &lt;li&gt;After the great success of usbnet(9) for USB ethernet drivers, core and I agreed to do the same for wifi - the result is called usbwifi(9) and makes conversion of usb drivers a lot easier than other wifi drivers. See &lt;a href=&quot;https://wiki.NetBSD.org/tutorials/converting_usb_drivers_to_usbwifi__40__9__41__/&quot;&gt;the conversion instructions&lt;/a&gt; for more details. usbwifi(9) is both quite similar but also quite different to usbnet(9), mostly for two reasons: it interfaces to a totally different stack, and many usb wlan chipsets are more complex than ethernet chipsets (e.g. have support for multiple send queues with different priorities). Developing usbwifi did cost quit some time (initially unplanned), but is expected to amortize over the next few drivers and quickly end up as a net win.&lt;/li&gt;
272 &lt;li&gt;I have been hitting a bug in the urtwn(4) driver used for inial usbwifi(9) development and still not found it (as of today). It seems to hit randomly and not be caused by the usbwifi(9) conversion - a fact that I found out only recently. So for now I will put this driver aside (after spending *way* too much time on it) and instead work on other usb drivers, returning to the bug every now and then and see if I can spot it. Maybe I can borrow a USB analyzer and get more insight that way.&lt;/li&gt;
273 &lt;/ul&gt;
274
275 &lt;p&gt;The current state of driver conversion and what drivers are still open are listed in &lt;a href=&quot;https://wiki.NetBSD.org/Driver_state_matrix/&quot;&gt;the wifi driver conversion matrix&lt;/a&gt;.&lt;/p&gt;
276
277 &lt;p&gt;Next steps ahead are:&lt;/p&gt;
278 &lt;ul&gt;
279 &lt;li&gt;&lt;s&gt;make another pass over documentation and improve things / fixup for recent changes&lt;/s&gt; (done before this blog post got published)&lt;/li&gt;
280 &lt;li&gt;sync the branch with HEAD and keep tracking it more closely&lt;/li&gt;
281 &lt;li&gt;convert run(4) to usbwifi&lt;/li&gt;
282 &lt;li&gt;revisit rtwn(4) and decide if/how it should be merged with urtwn(4)&lt;/li&gt;
283 &lt;li&gt;revisit iwm(4) and make it work fully&lt;/li&gt;
284 &lt;li&gt;convert all other drivers, starting with the ones I have hardware for&lt;/li&gt;
285 &lt;/ul&gt;
286
287 &lt;p&gt;Currently it is not clear if this branch can be merged to HEAD before branching for netbsd-10. We will not delay the netbsd-10 branch for this.&lt;/p&gt;
288 </content>
289 </entry>
290 <entry>
291 <id>https://blog.netbsd.org/tnf/entry/support_for_chdir_2_in</id>
292 <title type="html">Support for chdir(2) in posix_spawn(3)</title>
293 <author><name>martin</name></author>
294 <link rel="alternate" type="text/html" href="https://blog.netbsd.org/tnf/entry/support_for_chdir_2_in"/>
295 <published>2021-06-10T15:28:05+00:00</published>
296 <updated>2021-06-10T15:28:05+00:00</updated>
297 <category term="/Development" label="Development" />
298 <category term="gsoc" scheme="http://roller.apache.org/ns/tags/" />
299 <category term="posix_spawn" scheme="http://roller.apache.org/ns/tags/" />
300 <summary type="html">&lt;p&gt;Piyush Sachdeva is working on an extension to NetBSD&apos;s posix_spawn system call implementation and library support.&lt;/p&gt;
301
302 &lt;p&gt;He applied as a GSoC student, but unfortunately we only got a single slot from Google this year, so The NetBSD Foundation
303 offered Piyush to work on it by TNF funding outside of the official GSoC.&lt;/p&gt;
304
305 &lt;p&gt;In this post Piyush introduces himself and the project. He already started with the work... &lt;/p&gt;</summary>
306 <content type="html">&lt;h3&gt;This post was written by Piyush Sachdeva:&lt;/h3&gt;
307
308 &lt;p&gt;What really happens when you double click an icon on your desktop?&lt;/p&gt;
309
310 &lt;h1&gt;Support for chdir(2) in posix_spawn(3)&lt;/h1&gt;
311
312 &lt;p&gt;Processes are the bread and butter of your operating system. The moment
313 you double click an icon, that particular program gets loaded in your
314 Random Access Memory (RAM) and your operating system starts to run it. At
315 this moment the program becomes a process. Though you can only see the execution
316 of your process, the operating system (the Kernel) is always running a lot
317 of processes in the background to facilitate you.&lt;/p&gt;
318
319 &lt;p&gt;From the moment you hit that power button, everything that happens on the
320 screen is the result of some or the other process. In this post we are
321 going to talk about one such interface which helps in creation of your
322 programs.&lt;/p&gt;
323
324 &lt;h2&gt;The fork() &amp; exec() shenanigans&lt;/h2&gt;
325
326 &lt;p&gt;The moment a computer system comes alive, it launches a bunch of
327 processes. For the purpose of this blog let&amp;#x2019s call them, &amp;#x2018the master
328 processes&amp;#x2019. These processes run in perpetuity, provided the computer is
329 switched on. One such process is &lt;a href=&quot;#note1&quot;&gt;init/systemd/launchd (depending on your OS)&lt;/a&gt;.
330 This &amp;#x2018init&amp;#x2019 master process owns all the other processes in the computer,
331 either directly or indirectly.&lt;/p&gt;
332
333 &lt;p&gt;Operating systems are elegant, majestic software that work
334 seamlessly under the hood. They do so much without even breaking a sweat
335 (unless it&amp;#x2019s Windows). Let&apos;s consider a scenario where you have decided to
336 take a trip down memory lane and burst open those old photos. The &amp;#x2018init
337 master process&amp;#x2019 just can&amp;#x2019t terminate itself and let you look at your
338 photos. What if you unknowingly open a malicious file, which corrupts all
339 your data? So init doesn&amp;#x2019t just exit, rather it employs fork() and exec()
340 to start a new process. The fork() function is used to create child
341 processes which are an exact copy of their parents. Whichever process calls
342 fork, gets duplicated. The newly created process becomes the child of the
343 original running process and the original running process is called the
344 parent. Just how parents look after their kids, the parent process makes
345 sure that the child process doesn&apos;t do any mischief. So now you have two
346 exactly similar processes running in your computer.&lt;/p&gt;
347
348 &lt;img id=&quot;flowchar&quot;, src=&quot;//www.NetBSD.org/~martin/flowchart.jpg&quot;, alt=&quot;Flowchart of fork() + exec()&quot;, width=&quot;960&quot;, height=&quot;362&quot;, object-fit=&quot;contain&quot;&gt;
349
350 &lt;p&gt;One might think that the newly created child process doesn&amp;#x2019t really help
351 us. But actually, it does. Now exec() comes into the picture. What exec()
352 does is, it replaces any process which calls it. So what if we replace the child
353 process, the one we just thought to be useless, with our photos? That&apos;s
354 exactly what we are going to do indeed. This will result in replacement of
355 the fork() created child process with your photos. Therefore, the master
356 init process is still running and you can also enjoy your photos with no
357 threat to your data.&lt;/p&gt;
358
359
360 &lt;cite&gt;
361 &amp;#x201cNeither abstraction nor simplicity is a substitute for getting it right.
362 Sometimes, you just have to do the right thing, and when you do, it is way
363 better than the alternatives. There are lots of ways to design APIs for
364 process creation; however, the combination of fork() and exec() is simple
365 and immensely powerful. Here, the UNIX designers simply got it right.&amp;#x201d
366 &lt;/cite&gt; &lt;a href=&quot;#note2&quot;&gt;Lampson&amp;#x2019s Law - Getting it Right&lt;/a&gt;
367
368 &lt;p&gt;Now you could ask me, `But what about the title, some &amp;#x2018posix_spawn()&amp;#x2019
369 thing?&#180; Don&amp;#x2019t worry, that&amp;#x2019s next.&lt;/p&gt;
370
371 &lt;h2&gt;posix_spawn()&lt;/h2&gt;
372
373 &lt;p&gt;posix_spawn() is an alternative to the fork() + exec() routine. It
374 implements fork() and exec(), but not directly (as that would make it
375 slow, and we all need everything to be lightning fast). What actually
376 happens is that posix_spawn() only implements the functionality of the
377 fork() + exec() routines, but in one single call. However, because fork() +
378 exec() is a combination of two different calls, there is a lot of room for
379 customization. Whatever software you are running on your computer, calls
380 these routines on its own and does the necessary. Meanwhile a lot is
381 cooking in the background. Between the call to fork() and exec() there is
382 plenty of leeway for tweaking different aspects of the exec-ing process.
383 But posix_spawn doesn&amp;#x2019t bear this flexibility and therefore has a lot of
384 limitations. It does take a lot of parameters to give the caller some
385 flexibility, but it is not enough.&lt;/p&gt;
386
387 &lt;p&gt;Now the question before us is,
388 &amp;#x201cIf fork() + exec() is so much more powerful, then why have,
389 or use the posix_spawn() routine?&amp;#x201d The answer to that is, that
390 &lt;a href=&quot;#note3&quot;&gt;fork() and exec()&lt;/a&gt; are UNIX system routines.
391 They are not present in operating systems that are not a derivative of UNIX.
392 Eg- Windows implements a family of spawn functions.&lt;br/&gt;
393 There is another issue with fork() (not exec() ), which in reality is one
394 of the biggest reasons behind the growth of posix_spawn(). The outline of
395 the issue is that, creating child processes in multi-threaded programs is a
396 whole another ball game altogether.&lt;/p&gt;
397
398
399 &lt;p&gt;Concurrency is one of those disciplines in operating systems where the
400 order in which the cards are going to unravel is not always how you expect
401 them to. Multi-threading in a program is a way to do different and independent tasks of a
402 program simultaneously, to save time. No matter how jazzy or intelligent the above statement looks,
403 multi-threaded programs require an eagle&amp;#x2019s eye as they can often have a lot
404 of holes. Though the &amp;#x201ctasks&amp;#x201d are different and independent, they often
405 share a few common attributes. When these different tasks due to
406 concurrency start running in parallel, a data race begins to access those
407 shared attributes. To not wreak havoc, there are mechanisms through which,
408 when modifying/accessing these common attributes (Critical Section) we can
409 provide a sort of mutual exclusion (locks/conditional variables) - only
410 letting one of the processes modify the shared attribute at a time. Here
411 when things are already so intricate due to multithreading, and to top it
412 off, we start creating child processes. Complications are bound to arise.
413 When one of the threads from the multi-threaded program calls fork() to
414 create a child process, fork() does clone everything (variables, their
415 states, functions, etc) but it fails to clone other threads (though this is not
416 required at all times).&lt;/p&gt;
417
418 &lt;p&gt;The child process now knows only about that one thread which called fork().
419 But all the other attributes of the child that were inherited from
420 the parent (locks, mutexes) are set from the parent&amp;#x2019s address space
421 (considering multiple threads). So there is no way for the child process to
422 know which attributes conform to which parts of the parent. Also, those
423 mechanisms that we used to provide mutual exclusion, like locks and
424 conditional variables, need to be reset. This reset step is essential
425 in letting the parent access it&amp;#x2019s attributes. Failing this reset can cause deadlocks.
426 To put it simply, you can see how difficult things
427 have become all of a sudden. The posix_spawn() call is free from these
428 limitations of fork() encountered in multi-threaded programs. However, as
429 mentioned by me earlier, there needs to be enough rope to meet all the
430 requirements before posix_spawn() does the implicit exec().&lt;/p&gt;
431
432
433 &lt;h2&gt;About my Project&lt;/h2&gt;
434
435 &lt;p&gt;Hi, I am Piyush Sachdeva and I am going to start a project which will focus
436 on relaxing one limitation of posix_spawn - changing the current directory
437 of the child process, before the said call to exec() is made. This is not
438 going to restrict it to the parent&amp;#x2019s current working directory. Just
439 passing the new directory as one of the parameters will do the trick.
440 Resolving all the impediments would definitely be marvelous. Alas! That is
441 not possible. Every attempt to resolve even a single hindrance can create
442 plenty of new challenges.&lt;/p&gt;
443
444 &lt;p&gt;As already mentioned by me, posix_spawn() is a POSIX standard. Hence the
445 effect of my project will probably be reflected in the next POSIX release.
446 I came across this project through Google Summer of Code 2021. It was being
447 offered by The NetBSD Foundation Inc. However, as the slots for
448 Google Summer of Code were numbered, my project didn&amp;#x2019t make the selection.
449 Nevertheless, the Core Team at The NetBSD Foundation offered me to work on
450 the project and even extended a handsome stipend. I will forever be
451 grateful to The NetBSD Foundation for this opportunity.&lt;/p&gt;
452
453 &lt;h2&gt;Notes&lt;/h2&gt;
454
455 &lt;ul&gt;
456 &lt;li&gt;init, systemd &amp; launchd are system daemon processes. init is the
457 historical process present since System III UNIX. systemd was the replacement
458 for the authentic init which was written for the Linux kernel.
459 launchd is the MacOS alternative of init/systemd.&lt;/li&gt;
460
461 &lt;li&gt;This is taken from Operating Systems: The Three Easy Pieces book by
462 Andrea C. Arpaci-Dusseau and Remzi H. Arpaci-Dusseau.&lt;/li&gt;
463
464 &lt;li&gt;UNIX is the original AT&amp;T UNIX operating system developed at the Bell
465 Labs research center, headed by Ken Thompson and Dennis Ritchie.&lt;/li&gt;
466 &lt;/ul&gt;
467
468 &lt;h2&gt;References&lt;/h2&gt;
469
470 &lt;ol&gt;
471 &lt;li&gt; &lt;a name=&quot;note1&quot;&gt; Operating Systems: Three Easy Pieces by Andrea C. Arpaci-Dusseau and
472 Remzi H. Arpaci-Dusseau.&lt;/a&gt;&lt;/li&gt;
473 &lt;li&gt; &lt;a name=&quot;note2&quot;&gt; Advanced Programming in the UNIX Environment by W. Richard Stevens
474 and Stephen A. Rago.&lt;/a&gt;&lt;/li&gt;
475 &lt;li&gt; &lt;a name=&quot;note3&quot;&gt;UNIX and Linux System Administration Handbook by Evi Nemeth, Garth
476 Synder, Trent R. Hein, Ben Whaley and Dan Mackin.&lt;/a&gt;&lt;/li&gt;
477 &lt;/ol&gt;</content>
478 </entry>
479 <entry>
480 <id>https://blog.netbsd.org/tnf/entry/public_netbsd_irc_channels_moved</id>
481 <title type="html">Public NetBSD IRC chat channels moved to Libera</title>
482 <author><name>Nia Alarie</name></author>
483 <link rel="alternate" type="text/html" href="https://blog.netbsd.org/tnf/entry/public_netbsd_irc_channels_moved"/>
484 <published>2021-05-30T18:23:29+00:00</published>
485 <updated>2021-05-30T18:24:49+00:00</updated>
486 <category term="/General" label="General" />
487 <summary type="html">&lt;pHi everyone,&lt;/p&gt;
488
489 &lt;p&gt;Due to the unfortunate situation regarding changes in administration on
490 freenode.net, and the resulting chaos, we have decided to move the public
491 NetBSD IRC channels from freenode to irc.libera.chat.&lt;/p&gt;
492
493 &lt;p&gt;This includes:&lt;/p&gt;
494
495 &lt;ul&gt;
496 &lt;li&gt;&lt;a href=&quot;https://web.libera.chat/#NetBSD&quot;&gt;#NetBSD&lt;/a&gt; - general discussion&lt;/li&gt;
497 &lt;li&gt;&lt;a href=&quot;https://web.libera.chat/#netbsd-code&quot;&gt;#netbsd-code&lt;/a&gt; - development discussion&lt;/li&gt;
498 &lt;li&gt;&lt;a href=&quot;https://web.libera.chat/#pkgsrc&quot;&gt;#pkgsrc&lt;/a&gt; - pkgsrc (primarily development) discussion&lt;/li&gt;
499 &lt;/ul&gt;
500
501 &lt;p&gt;You can find information on connecting to Libera at &lt;a href=&quot;https://libera.chat&quot;&gt;https://libera.chat/&lt;/a&gt;&lt;/p&gt;</summary>
502 <content type="html">&lt;p&gt;Hi everyone,&lt;/p&gt;
503
504 &lt;p&gt;Due to the unfortunate situation regarding changes in administration on
505 freenode.net, and the resulting chaos, we have decided to move the public
506 NetBSD IRC chat channels from freenode to irc.libera.chat.&lt;/p&gt;
507
508 &lt;p&gt;This includes:&lt;/p&gt;
509
510 &lt;ul&gt;
511 &lt;li&gt;&lt;a href=&quot;https://web.libera.chat/#NetBSD&quot;&gt;#NetBSD&lt;/a&gt; - general discussion&lt;/li&gt;
512 &lt;li&gt;&lt;a href=&quot;https://web.libera.chat/#netbsd-code&quot;&gt;#netbsd-code&lt;/a&gt; - development discussion&lt;/li&gt;
513 &lt;li&gt;&lt;a href=&quot;https://web.libera.chat/#pkgsrc&quot;&gt;#pkgsrc&lt;/a&gt; - pkgsrc (primarily development) discussion&lt;/li&gt;
514 &lt;/ul&gt;
515
516 &lt;p&gt;You can find information on connecting to Libera at &lt;a href=&quot;https://libera.chat&quot;&gt;https://libera.chat/&lt;/a&gt;&lt;/p&gt;</content>
517 </entry>
518 <entry>
519 <id>https://blog.netbsd.org/tnf/entry/netbsd_9_2_released</id>
520 <title type="html">NetBSD 9.2 released</title>
521 <author><name>Nia Alarie</name></author>
522 <link rel="alternate" type="text/html" href="https://blog.netbsd.org/tnf/entry/netbsd_9_2_released"/>
523 <published>2021-05-17T14:06:03+00:00</published>
524 <updated>2021-05-17T14:37:04+00:00</updated>
525 <category term="/Release engineering" label="Release engineering" />
526 <summary type="html">&lt;p&gt; The NetBSD Project is pleased to announce NetBSD 9.2 &quot;Nakatomi Socrates&quot;, the second update of the NetBSD 9 release branch.&lt;/p&gt;
527
528 &lt;p&gt;As well as the usual bug, stability, and security fixes, this release includes: support for exporting ZFS filesystems over NFS, various updates to the bozotic HTTP daemon, improvements to ARM 32-bit and Linux compatibility, &lt;code&gt;fread()&lt;/code&gt; performance improvements, support for the TP-Link TL-WN821N V6 wireless adapter, support for the Allwinner H5 cryptographic accelerator, Pinebook Pro display brightness fixes, new defaults for &lt;code&gt;kern.maxfiles&lt;/code&gt;, and accessibility improvements for the default window manager configuration.&lt;/p&gt;
529
530 &lt;p&gt;&lt;a href=&quot;//www.NetBSD.org/releases/formal-9/NetBSD-9.2.html&quot;&gt;Release notes and download links for NetBSD 9.2&lt;/a&gt;&lt;/p&gt;</summary>
531 <content type="html">&lt;p&gt; The NetBSD Project is pleased to announce NetBSD 9.2 &quot;Nakatomi Socrates&quot;, the second update of the NetBSD 9 release branch.&lt;/p&gt;
532
533 &lt;p&gt;As well as the usual bug, stability, and security fixes, this release includes: support for exporting ZFS filesystems over NFS, various updates to the bozotic HTTP daemon, improvements to ARM 32-bit and Linux compatibility, &lt;code&gt;fread()&lt;/code&gt; performance improvements, support for the TP-Link TL-WN821N V6 wireless adapter, support for the Allwinner H5 cryptographic accelerator, Pinebook Pro display brightness fixes, new defaults for &lt;code&gt;kern.maxfiles&lt;/code&gt;, and accessibility improvements for the default window manager configuration.&lt;/p&gt;
534
535 &lt;p&gt;&lt;a href=&quot;//www.NetBSD.org/releases/formal-9/NetBSD-9.2.html&quot;&gt;Release notes and download links for NetBSD 9.2&lt;/a&gt;&lt;/p&gt;</content>
536 </entry>
537 <entry>
538 <id>https://blog.netbsd.org/tnf/entry/aiomixer_x_open_curses_and</id>
539 <title type="html">aiomixer, X/Open Curses and ncurses, and other news</title>
540 <author><name>Nia Alarie</name></author>
541 <link rel="alternate" type="text/html" href="https://blog.netbsd.org/tnf/entry/aiomixer_x_open_curses_and"/>
542 <published>2021-05-12T13:35:14+00:00</published>
543 <updated>2021-05-12T14:48:40+00:00</updated>
544 <category term="/Development" label="Development" />
545 <summary type="html">&lt;p&gt;aiomixer, X/Open Curses and ncurses, and other news&lt;/p&gt;</summary>
546 <content type="html">&lt;p&gt;
547 aiomixer is an application that I&apos;ve been maintaining outside of NetBSD for a
548 few years. It was available as a package, and was a &quot;graphical&quot; (curses,
549 terminal-based) mixer for NetBSD&apos;s audio API, inspired by programs like alsamixer.
550 For some time I&apos;ve thought that it should be integrated into the NetBSD
551 base system - it&apos;s small and simple, very useful, and many developers
552 and users had it installed (some told me that they would install it on all
553 of their machines that needed audio output).
554 For my particular use case, as well as my NetBSD laptop, I have some small
555 NetBSD machines around the house plugged into speakers that I play music from.
556 Sometimes I like to SSH into them to adjust the playback volume, and it&apos;s often
557 easier to do visually than with
558 &lt;a href=&quot;https://man.NetBSD.org/mixerctl.1&quot;&gt;mixerctl(1)&lt;/a&gt;.
559 &lt;/p&gt;
560
561 &lt;p&gt;
562 However, there was one problem: when I first wrote aiomixer 2 years ago,
563 I was intimidated by the curses API, so opted to use the
564 &lt;a href=&quot;https://invisible-island.net/cdk/&quot;&gt;Curses Development Kit&lt;/a&gt;
565 instead.
566 This turned out to be a mistake, as not only was CDK inflexible for an
567 application like aiomixer, it introduced a hard dependency on ncurses.
568 &lt;/p&gt;
569
570 &lt;h2&gt;X/Open Curses and ncurses&lt;/h2&gt;
571
572 &lt;p&gt;
573 Many people think ncurses is the canonical way to develop terminal-based
574 applications for Unix, but it&apos;s actually an implementation of the
575 &lt;a href=&quot;https://pubs.opengroup.org/onlinepubs/7908799/xcurses/curses.h.html&quot;&gt;X/Open Curses specification&lt;/a&gt;.
576 There&apos;s a few other Curses implementations:
577 &lt;/p&gt;
578
579 &lt;ul&gt;
580 &lt;li&gt;&lt;a href=&quot;https://man.netbsd.org/curses.3&quot;&gt;NetBSD libcurses&lt;/a&gt;&lt;/li&gt;
581 &lt;li&gt;&lt;a href=&quot;https://docs.oracle.com/cd/E36784_01/html/E36880/curses-3curses.html&quot;&gt;Solaris libcurses&lt;/a&gt;&lt;/li&gt;
582 &lt;li&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/PDCurses&quot;&gt;PDCurses&lt;/a&gt;, used on Windows&lt;/li&gt;
583 &lt;/ul&gt;
584
585 &lt;p&gt;
586 NetBSD curses is descended from the original BSD curses, but contains
587 many useful extensions from ncurses as well. We use it all over the
588 base system, and for most packages in pkgsrc.
589 It&apos;s also been
590 &lt;a href=&quot;https://github.com/sabotage-linux/netbsd-curses&quot;&gt;ported to other operating systems&lt;/a&gt;,
591 including Linux.
592 As far as I&apos;m aware, NetBSD is one of the last operating systems left
593 that doesn&apos;t primarily depend on ncurses.
594 &lt;/p&gt;
595
596 &lt;p&gt;
597 There&apos;s one crucial incompatibility, however: ncurses exposes its
598 internal data structures, NetBSD libcurses keeps them opaque.
599 Since CDK development is very tied to ncurses development (they have
600 the same maintainer), CDK peeks into those structures, and can&apos;t
601 be used with NetBSD libcurses.
602 There are also a few places where ncurses breaks with X/Open Curses,
603 like
604 &lt;a href=&quot;https://github.com/irssi/irssi/pull/1305&quot;&gt;this case I recently fixed in irssi&lt;/a&gt;.
605 &lt;/p&gt;
606
607 &lt;h2&gt;Rewriting aiomixer&lt;/h2&gt;
608
609 &lt;p&gt;
610 I was able to rewrite aiomixer in a few days using only my free time
611 and NetBSD libcurses. It&apos;s now been imported to the base system.
612 It was a good lesson in why Curses isn&apos;t actually that intimidating -
613 while there are many functions, they&apos;re mostly variations on the
614 same thing. Using Curses directly resulted in a much lighter and
615 more usable application, and provided a much better fit for the
616 types of widgets I needed.
617 &lt;/p&gt;
618
619 &lt;p&gt;
620 Many people also provided testing, and I learned a lot about
621 how different terminal attributes should be used in the process.
622 NetBSD is probably one of the few communities where you&apos;ll get
623 easy and direct feedback on how to not only make your software
624 work well in a variety of terminal emulators, but also old school
625 hardware terminals. During development, I was also able to find
626 a strange bug in the curses library&apos;s window resizing function.
627 &lt;/p&gt;
628
629 &lt;p&gt;
630 The API support was also improved, and the new version of aiomixer
631 should work better with a wider variety of sound hardware drivers.
632 &lt;/p&gt;
633
634 &lt;a href=&quot;https://cdn.netbsd.org/pub/NetBSD/misc/nia/aiomixer2.png&quot;&gt;&lt;img src=&quot;https://cdn.netbsd.org/pub/NetBSD/misc/nia/aiomixer2.png&quot; width=400&quot; /&gt;&lt;/a&gt;
635
636 &lt;h2&gt;Other happenings&lt;/h2&gt;
637
638 &lt;p&gt;
639 Since I&apos;m done plugging my own work, I thought I might talk
640 a bit about some other recent changes to CURRENT.
641 &lt;/p&gt;
642
643 &lt;ul&gt;
644 &lt;li&gt;Most ports of NetBSD now build with GCC 10, thanks to work by mrg.
645 The new version of GCC introduced many new compiler warnings. By
646 default, since NetBSD is compiled with a fixed toolchain version,
647 we use &lt;code&gt;-Werror&lt;/code&gt;. Many minor warnings and actual-bugs
648 were uncovered and fixed with the new compiler.&lt;/li&gt;
649 &lt;li&gt;On the ARM front, support for the Allwiner V3S system-on-a-chip
650 was introduced thanks to work by Rui-Xiang Guo. This is an
651 older model Allwinner core, primarily used on small embedded
652 devices. It&apos;s of likely interest to hardware hackers because it
653 comes in an easily soldered package. A development board is
654 available, the Lichee Pi Zero. Also in the Allwinner world,
655 support for the H3 SoC (including the NanoPi R1) was added
656 to the
657 &lt;a href=&quot;https://man.NetBSD.org/sun8icrypto.4&quot;&gt;sun8icrypto(4)&lt;/a&gt;
658 driver by bad.&lt;/li&gt;
659 &lt;li&gt;Support for RISC-V is progressing, including an UEFI
660 bootloader for 64-bit systems, and an in-kernel disassembler.
661 Some NetBSD developers have recently obtained Beagle-V development
662 boards.&lt;/li&gt;
663 &lt;li&gt;On the SPARC64 front, support for sun4v is progressing thanks
664 to work by palle. The
665 sun4v architecture includes most newer SPARC servers that are
666 based on the
667 &lt;a href=&quot;https://en.wikipedia.org/wiki/Oracle_VM_Server_for_SPARC&quot;&gt;Logical Domains&lt;/a&gt;
668 architecture - virtualization is
669 implemented at the hardware/firmware level, and operating systems
670 get an abstracted view of the underlying hardware. With other
671 operating systems are discussing removing support for SPARC64, there&apos;s
672 an interest among NetBSD developers in adding and maintaining
673 support for this very interesting hardware from Oracle, Fujitsu,
674 and Sun in an open source operating system, not just Oracle
675 Solaris.&lt;/li&gt;
676 &lt;li&gt;A kernel-wide audit and rework of the auto-configuration
677 APIs was completed by thorpej.&lt;/li&gt;
678 &lt;li&gt;Various new additions and fixes have been made to the
679 networking stack&apos;s
680 &lt;a href=&quot;https://man.NetBSD.org/pppoe.4&quot;&gt;PPP over Ethernet&lt;/a&gt;
681 support by yamaguchi.&lt;/li&gt;
682 &lt;li&gt;A new API was introduced by macallan that allows adding
683 a &lt;code&gt;-l&lt;/code&gt; option to the
684 &lt;a href=&quot;https://man.NetBSD.org/wsfontload.8&quot;&gt;wsfontload(8)&lt;/a&gt;
685 command, allowing easy viewing of the tty fonts currently
686 loaded into the kernel.&lt;/li&gt;
687 &lt;li&gt;... OK, I&apos;m not done plugging my own work: I recently
688 wrote new documentation on
689 using
690 &lt;a href=&quot;https://www.netbsd.org/docs/guide/en/chap-virt.html&quot;&gt;NetBSD for virtualization&lt;/a&gt;,
691 &lt;a href=&quot;https://www.netbsd.org/docs/guide/en/chap-power.html&quot;&gt;Power Management&lt;/a&gt;,
692 and rewrote the
693 &lt;a href=&quot;https://www.netbsd.org/docs/guide/en/index.html&quot;&gt;NetBSD Guide&lt;/a&gt;&apos;s sections on
694 &lt;a href=&quot;https://www.netbsd.org/docs/guide/en/chap-net-practice.html&quot;&gt;Networking in Practice&lt;/a&gt;
695 and
696 &lt;a href=&quot;https://www.netbsd.org/docs/guide/en/chap-audio.html&quot;&gt;Audio&lt;/a&gt;.
697 I also recently added support for the
698 &lt;a href=&quot;https://en.wikipedia.org/wiki/Neo_(keyboard_layout)&quot;&gt;Neo 2 keyboard layout&lt;/a&gt;
699 to NetBSD&apos;s console system - Neo 2 is a Dvorak-like
700 optimized layout for German and other languages based on
701 multiple layers for alphabetical characters, navigation,
702 and symbols.&lt;/li&gt;
703
704 &lt;/ul&gt;
705 </content>
706 </entry>
707 <entry>
708 <id>https://blog.netbsd.org/tnf/entry/gsoc_reports_make_system_31</id>
709 <title type="html">GSoC Reports: Make system(3), popen(3) and popenve(3) use posix_spawn(3) internally (Final report)</title>
710 <author><name>Nikita Ronja Gillmann</name></author>
711 <link rel="alternate" type="text/html" href="https://blog.netbsd.org/tnf/entry/gsoc_reports_make_system_31"/>
712 <published>2021-03-30T11:11:15+00:00</published>
713 <updated>2022-02-24T10:36:27+00:00</updated>
714 <category term="/The NetBSD Foundation" label="The NetBSD Foundation" />
715 <category term="gsoc" scheme="http://roller.apache.org/ns/tags/" />
716 <summary type="html">&lt;i&gt;This report was prepared by Nikita Ronja Gillmann as a part of Google Summer of Code 2020&lt;/i&gt;
717
718 &lt;p&gt;This is my second and final report for the Google Summer of Code project I am working on for NetBSD.&lt;/p&gt;
719
720 &lt;p&gt;
721 My code can be found at &lt;a href=&quot;https://github.com/teknokatze/src/&quot;&gt;github.com/teknokatze/src&lt;/a&gt; in the gsoc2020 branch, at the time of writing some of it is still missing. The test facilities and logs can be found in &lt;a href=&quot;https://github.com/teknokatze/gsoc2020/&quot;&gt;github.com/teknokatze/gsoc2020&lt;/a&gt;. A diff can be found at &lt;a href=&quot;https://github.com/NetBSD/src/compare/trunk...teknokatze:gsoc2020&quot;&gt;github&lt;/a&gt; which will later be split into several patches before it is sent to QA for merging.
722 &lt;/p&gt;
723
724 &lt;p&gt;
725 The initial and defined goal of this project was to &lt;a href=&quot;https://blog.netbsd.org/tnf/entry/gsoc_reports_make_system_3&quot;&gt;make system(3) and popen(3) use posix_spawn(3) internally&lt;/a&gt;, which had been completed in June.
726 For the second part I was given the task to replace fork+exec calls in our standard shell (sh) in one scenario. Similar to the previous goal we determine through implementation if the initial motivation, to get performance improvements, is correct otherwise we collect metrics for why posix_spawn() in this case should be avoided. This second part meant in practice that I had to add and change code in the kernel, add a new public libc function, and understand shell internals.
727 &lt;/p&gt;
728 </summary>
729 <content type="html">&lt;i&gt;This report was prepared by Nikita Ronja Gillmann as a part of Google Summer of Code 2020&lt;/i&gt;
730
731 &lt;p&gt;This is my second and final report for the Google Summer of Code project I am working on for NetBSD.&lt;/p&gt;
732
733 &lt;p&gt;
734 My code can be found at &lt;a href=&quot;https://github.com/nikicoon/src/&quot;&gt;github.com//src&lt;/a&gt; in the &lt;i&gt;gsoc2020&lt;/i&gt; branch, at the time of writing some of it is still missing.
735 The test facilities and logs can be found in &lt;a href=&quot;https://github.com/nikicoon/gsoc2020/&quot;&gt;github.com/nikicoon/gsoc2020&lt;/a&gt;.
736 A diff can be found at &lt;a href=&quot;https://github.com/NetBSD/src/compare/trunk...nikicoon:gsoc2020&quot;&gt;github&lt;/a&gt; which will later be split into several patches before it is sent to QA for merging.
737 &lt;/p&gt;
738
739 &lt;p&gt;
740 The initial and defined goal of this project was to &lt;a href=&quot;https://blog.netbsd.org/tnf/entry/gsoc_reports_make_system_3&quot;&gt;make system(3) and popen(3) use posix_spawn(3) internally&lt;/a&gt;, which had been completed in June.
741 For the second part I was given the task to replace fork+exec calls in our standard shell (sh) in one scenario. Similar to the previous goal we determined through implementation if the initial motivation, to get performance improvements, is correct otherwise we collect metrics for why posix_spawn() in this case should be avoided.
742 This second part meant in practice that I had to add and change code in the kernel, add a new public libc function, and understand shell internals.
743 &lt;/p&gt;
744
745 &lt;h2&gt;Summary of part 1&lt;/h2&gt;
746 &lt;p&gt;
747 Prior work: In GSoC 2012 Charles Zhang &lt;a href=&quot;https://blog.netbsd.org/tnf/entry/posix_spawn_syscall_added&quot;&gt;added the posix_spawn syscall&lt;/a&gt; which according to &lt;a href=&quot;https://web.archive.org/web/20150905210045/http://netbsd-soc.sourceforge.net/projects/posix_spawn/&quot;&gt;its SF repository&lt;/a&gt; at the time (maybe even now, I have not looked very much into comparing all other systems and libcs + kernels) is an in-kernel implementation of posix_spawn which provides performance benefits compared to FreeBSD and other systems which had a userspace implementation (in 2012).
748 &lt;/p&gt;
749 &lt;p&gt;
750 After 1 week of reading POSIX and writing code, 2 weeks of coding and another 1.5 weeks of bugfixes I have successfully implemented posix_spawn in usage in system(3) and popen(3) internally.&lt;/p&gt;&lt;p&gt;The biggest challenge for me was to understand POSIX, to read the standard. I am used to reading more formal books, but I can&apos;t remember working with POSIX Standard directly before.
751 &lt;/p&gt;
752 &lt;h3&gt;system(3)&lt;/h3&gt;
753 &lt;p&gt;system(3) was changed to use posix_spawnattr_ (where we used sigaction before) and posix_spawn (which replaced execve + vfork calls).&lt;/p&gt;
754 &lt;h3&gt;popen(3) and popenve(3)&lt;/h3&gt;
755 &lt;p&gt;
756 Since the popen and popenve implementation in NetBSD&apos;s libc use a couple of shared helper functions, I was able to change both functions while keeping the majority of the changes focused on (some of ) the helper functions (pdes_child).
757 &lt;/p&gt;
758 &lt;p&gt;
759 pdes_child, an internal function in popen.c, now takes one more argument (&lt;i&gt;const char *cmd&lt;/i&gt;) for the command to pass to posix_spawn which is called in pdes_child.
760 &lt;/p&gt;
761 &lt;p&gt;
762 On a high level what happens in pdes_child() and popen is that we first lock the pidlist_mutex. Then we create a file file action list for all concurrent popen() / popenve() instances and the side of the pipe not necessary, and the move to stdin/stdout. We unlock the pidlist_mutex. Finally we return the list and destroy.
763 &lt;/p&gt;
764 &lt;p&gt;
765 In the new version of this helper function which now handles the majority of what popen/popenve did, we have to initialize a file_actions object which by default contains no file actions for posix_spawn() to perform. Since we have to have error handling and a common return value for the functions calling pdes_child() and deconstruction, we make use of &lt;i&gt;goto&lt;/i&gt; in some parts of this function.
766 &lt;/p&gt;
767 &lt;p&gt;
768 The close() and dup2() actions now get replaced by corresponding file_actions syscalls, they are used to specify a series of actions to be performed by a posix_spawn operation.&lt;/p&gt;
769 &lt;p&gt;After this series of actions, we call _readlockenv(), and call posix_spawn with the file_action object and the other arguments to be executed. If it succeeds, we return the pid of the child to popen, otherwise we return -1, in both cases we destroy the file_action object before we proceed.&lt;/p&gt;
770 &lt;p&gt;
771 In popen and popenve our code has been reduced to the &lt;i&gt;pid == -1&lt;/i&gt; branch, everything else happens in pdes_child() now.
772 &lt;/p&gt;
773 &lt;p&gt;
774 After readlockenv we call pdes_child and pass it the command to execute in the posix_spawn&apos;d child process; if pdes_child returns -1 we run the old error handling code. Likewise for popenve.&lt;/p&gt;
775 &lt;p&gt;
776 The outcome of the first part is, that thanks to how we implement posix_spawn in NetBSD we reduced the syscalls being made for popen and system.
777 A full test with proper timing should indicate this, my reading was based on comparing old and new logs with ktrace and kdump.
778 &lt;/p&gt;
779 &lt;h2&gt;sh, posix_spawn actions, libc and kernel - Part 2&lt;/h2&gt;
780 &lt;h3&gt;Motivation&lt;/h3&gt;
781 &lt;p&gt;
782 The main goal of part 2 of this project was to change sh(1) to
783 determine which simple cases of (v)fork + exec I could replace, and to
784 replace them with posix_spawn where it makes sense.&lt;/p&gt;
785 &lt;p&gt;
786 fork needs to create a new address space by cloning the address space,
787 or in the case of vfork update at least some reference counts.
788 posix_spawn can avoid most of this as it creates the new address space from scratch.
789 &lt;/p&gt;
790 &lt;h3&gt;Issues&lt;/h3&gt;
791 &lt;p&gt;
792 The current posix_spawn as defined in POSIX has no good way to do tcsetpgrp, and we found
793 that &lt;a href=&quot;https://github.com/fish-shell/fish-shell/issues/360&quot;&gt;fish just avoids posix_spawn for foreground processes&lt;/a&gt;.
794 &lt;/p&gt;
795 &lt;h3&gt;Implementation&lt;/h3&gt;
796 &lt;p&gt;
797 Since, roughly speaking, modern BSDs handle &amp;quot;#!&amp;quot; execution in the kernel (probably since before the 1990s, systems which didn&apos;t handle this started to disappear most likely in the mid to late 90s), our main concern so far was in the evalcmd function the default cmd switch case (&apos;NORMALCMD&apos;).&lt;/p&gt;&lt;p&gt;After adjusting the function to use posix_spawn, I hit an issue in the execution of the curses application htop where htop would run but input would not be accepted properly (keysequences pressed are visible).
798 In pre-posix_spawn sh, every subprocess that sh (v)forked runs forkchild() to set up the subprocess&apos;s environment.
799 With posix_spawn, we need to arrange posix_spawn actions to do the same thing.
800 &lt;/p&gt;
801 &lt;p&gt;
802 The intermediate resolution was to switch FORK_FG processes to fork+exec again. For foreground processes with job control we&apos;re in an interactive shell, so the performance benefit is small enough in this case to be negligible. It&apos;s really only for shell scripts that it matters.&lt;/p&gt;&lt;p&gt;Next I implemented a posix_spawn file_action, with the prototype
803 &lt;/p&gt;
804 &lt;pre&gt;
805 &lt;code&gt;int posix_spawn_file_actions_addtcsetpgrp(posix_spawn_file_actions_t *fa, int fildes)&lt;/code&gt;
806 &lt;/pre&gt;
807 &lt;p&gt;
808 The kernel part of this was implemented inline in sys/kern/kern_exec.c, in the function handle_posix_spawn_file_actions() for the new case &apos;FAE_TCSETPGRP&apos;.&lt;/p&gt;&lt;p&gt;The new version of the code is still in testing and debugging phase and at the time of writing not included in my repository (it will be published after Google Summer of Code when I&apos;m done moving).
809 &lt;/p&gt;
810 &lt;h3&gt;Future steps&lt;/h3&gt;
811 &lt;h4&gt;posix_spawnp kernel implementation&lt;/h4&gt;
812 &lt;p&gt;
813 According to a conversation with kre@, the posix_spawnp() implementation we have is just itterating over $PATH calling posix_spawn until it succeeds.
814 For some changes we might want a kernel implementation of posix_spawnp(), as the path search is supposed to happen in the kernel so the file actions are only ever run once:
815 &lt;/p&gt;
816 &lt;pre&gt;
817 &lt;code&gt;
818 some of the file actions may be &amp;quot;execute once only&amp;quot;,
819 they can&apos;t be repeated (eg: handling &amp;quot;set -C; cat foo &amp;gt;file&amp;quot; - file
820 can only be created once, that has to happen before the exec (as the fd
821 needs to be made stdout), and then the exec part of posix_spawn is
822 attempted - if that fails, when it can&apos;t find &amp;quot;cat&amp;quot; in $HOME/bin (or
823 whatever is first in $PATH) and we move along to the next entry (maybe /bin
824 doesn&apos;t really matter) then the repeated file action fails, as file now
825 exists, and &amp;quot;set -C&amp;quot; demands that we cannot open an already existing file
826 (noclobber mode). It would be nice for this if there were &amp;quot;clean up on
827 failure&amp;quot; actions, but that is likely to be very difficult to get right,
828 and each would need to be attached to a file action, so only those which
829 had been performed would result in cleanup attempts.
830 &lt;/code&gt;
831 &lt;/pre&gt;
832 &lt;h4&gt;Replacing all of fork+exec in sh&lt;/h4&gt;
833 &lt;p&gt;
834 Ideally we could replace all of (v)fork + exec with posix_spawn.
835 According to my mentors there is pmap synchronisation as an impact of
836 constructing the vm space from scratch with (v)fork.
837 Less IPIs (inter-processor interrupts) matter for small processes too.
838 &lt;/p&gt;
839 &lt;h4&gt;posix_spawn_file_action_ioctl&lt;/h4&gt;
840 &lt;p&gt;
841 Future directions could involve a posix_spawn action for an arbitrary ioctl.
842 &lt;/p&gt;
843 &lt;h2&gt;Thanks&lt;/h2&gt;
844 &lt;p&gt;
845 My thanks go to fellow NetBSD developers for answering questions, most recently kre@ for sharing invaluable sh knowledge, Riastradh and J&#246;rg as the mentors I&apos;ve interacted with most of the time and for their often in-depth explanations as well as allowing me to ask questions I sometimes felt were too obvious. My friends, for sticking up with my &amp;quot;weird&amp;quot; working schedule. Lastly would like to thank the Google Summer of Code program for continuing through the ongoing pandemic and giving students the chance to work on projects full-time.
846 &lt;/p&gt;</content>
847 </entry>
848 <entry>
849 <id>https://blog.netbsd.org/tnf/entry/hitting_donation_milestone_financial_report</id>
850 <title type="html">Hitting donation milestone, financial report for 2020</title>
851 <author><name>Maya Rashish</name></author>
852 <link rel="alternate" type="text/html" href="https://blog.netbsd.org/tnf/entry/hitting_donation_milestone_financial_report"/>
853 <published>2021-03-29T09:17:29+00:00</published>
854 <updated>2021-03-29T10:32:22+00:00</updated>
855 <category term="/The NetBSD Foundation" label="The NetBSD Foundation" />
856 <summary type="html">&lt;p&gt;
857 We nearly hit our donation milestone set after the release of 9.0 of $50,000.&lt;br /&gt;
858 These donations have enabled us to fund significant paid work on NetBSD in 2020.
859 &lt;/p&gt;</summary>
860 <content type="html">&lt;p&gt;
861 We nearly hit our 2020 donation milestone set after the release of 9.0 of $50,000.
862 These donations have enabled us to fund significant work on NetBSD in 2020 such as:
863 &lt;ul&gt;
864 &lt;li&gt;an aarch64 package build server, &lt;a href=&quot;http://victory.netbsd.org/pkgsrc/packages/reports/HEAD/evbarm64-9.0/&quot;&gt;victory.netbsd.org&lt;/a&gt;. Thanks to Western Washington University for hosting this machine.&lt;/li&gt;
865 &lt;li&gt;&lt;a href=&quot;https://blog.netbsd.org/tnf/entry/wifi_renewal_restarted&quot;&gt;Modernizing wi-fi network stack&lt;/a&gt; and release engineering work by Martin Husemann&lt;/li&gt;
866 &lt;li&gt;&lt;a href=&quot;https://blog.netbsd.org/tnf/entry/lldb_work_concluded&quot;&gt;LLDB support&lt;/a&gt; by Micha&#322; G&#243;rny&lt;/li&gt;
867 &lt;li&gt;&lt;a href=&quot;https://blog.netbsd.org/tnf/entry/accomplishment_of_porting_ptrace_2&quot;&gt;ptrace and GDB improvements&lt;/a&gt; by Kamil Rytarowski&lt;/li&gt;
868 &lt;/ul&gt;
869 &lt;/p&gt;
870
871 &lt;p&gt;
872 If you are interested in seeing more work like this, please consider donating &lt;a href=&quot;https://www.paypal.com/cgi-bin/webscr?cmd=_donations&amp;business=paypal%40NetBSD.org&amp;currency_code=USD&amp;source=url&quot;&gt;via PayPal&lt;/a&gt; or &lt;a href=&quot;https://github.com/sponsors/NetBSD&quot;&gt;GitHub sponsors&lt;/a&gt;.
873 &lt;/p&gt;
874
875 &lt;p&gt;
876 The &lt;a href=&quot;https://www.netbsd.org/foundation/reports/financial/2020.html&quot;&gt;financial report for 2020&lt;/a&gt; is also available.
877 &lt;/p&gt;
878
879 &lt;p&gt;
880 Note: We realize that this data is inconsistent with the website indicator of donations. This is due to the fact the website is updated manually in an error-prone process as the donations are processed. The financial report (just completed) is prepared separately using &lt;a href=&quot;https://www.ledger-cli.org/&quot;&gt;ledger&lt;/a&gt;.
881 &lt;/p&gt;</content>
882 </entry>
883 <entry>
884 <id>https://blog.netbsd.org/tnf/entry/allen_k_briggs_memorial_scholarship</id>
885 <title type="html">Allen K. Briggs Memorial Scholarship</title>
886 <author><name>Leonardo Taccari</name></author>
887 <link rel="alternate" type="text/html" href="https://blog.netbsd.org/tnf/entry/allen_k_briggs_memorial_scholarship"/>
888 <published>2020-12-21T11:37:59+00:00</published>
889 <updated>2020-12-21T11:37:59+00:00</updated>
890 <category term="/General" label="General" />
891 <content type="html">&lt;p&gt;
892 Allen Briggs was one of the earliest members of the NetBSD community,
893 pursuing his interest in macBSD, and moving to become a NetBSD
894 developer when the two projects merged. Allen was known for his
895 quiet and relaxed manner, and always brought a keen wisdom with
896 him; allied with his acute technical expertise, he was one of the
897 most valued members of the NetBSD community.
898 &lt;/p&gt;
899
900 &lt;p&gt;
901 He was a revered member of the NetBSD core team, and keenly involved
902 in many aspects of its application; from working on ARM chips to
903 helping architect many projects, Allen was renowned for his expertise.
904 He was a distinguished engineer at Apple, and used his NetBSD
905 expertise there to bring products to market.
906 &lt;/p&gt;
907
908 &lt;p&gt;
909 Allen lived in Blacksburg Virginia with his wife and twin boys and
910 was active with various community volunteer groups. His family
911 touched the families of many other NetBSD developers and those
912 friendships have endured beyond his passing.
913 &lt;/p&gt;
914
915 &lt;p&gt;
916 We have received the following from Allen&apos;s family and decided to
917 share it with the NetBSD community. If you can, we would ask you
918 to consider contributing to his Memorial Scholarship.
919 &lt;/p&gt;
920
921 &lt;p&gt;
922 &lt;a href=&quot;https://www.ncssm.edu/donate/distance-education/allen-k-briggs-88-memorial-scholarship&quot;&gt;https://www.ncssm.edu/donate/distance-education/allen-k-briggs-88-memorial-scholarship&lt;/a&gt;
923 &lt;/p&gt;
924
925 &lt;p&gt;
926 The Allen K. Briggs Memorial Scholarship is an endowment to provide
927 scholarships in perpetuity for summer programs at the North Carolina
928 School of Science &amp; Math, which Allen considered to be a place that
929 fundamentally shaped him as a person. We would love to invite
930 Allen&apos;s friends and colleagues from the BSD community to donate to
931 this cause so that we can provide more scholarships to students
932 with financial need each year. We are approximately halfway to our
933 goal of $50K with aspirations to exceed that target and fund
934 additional scholarships.
935 &lt;/p&gt;
936
937 &lt;p&gt;
938 Two quick notes on donating: &lt;strong&gt;Important!&lt;/strong&gt; When donating, you must
939 select &quot;Allen K. Briggs Memorial Scholarship&quot; under designation
940 for the donation to be routed to the scholarship If you have the
941 option to use employer matching (i.e., donating to NCSSM through
942 an employer portal to secure a match from your employer), please
943 email the NCSSM Foundation&apos;s Director of Development, April Horton
944 (&lt;code&gt;april.horton@ncssm.edu&lt;/code&gt;), after donating to let her know you want
945 your gift and employer match to go to the Allen K. Briggs Memorial
946 Scholarship Thanks in advance for your help. I&apos;d be happy to answer
947 any questions you or any others have about this.
948 &lt;/p&gt;
949 </content>
950 </entry>
951 <entry>
952 <id>https://blog.netbsd.org/tnf/entry/netbsd_9_1_released</id>
953 <title type="html">NetBSD 9.1 released</title>
954 <author><name>martin</name></author>
955 <link rel="alternate" type="text/html" href="https://blog.netbsd.org/tnf/entry/netbsd_9_1_released"/>
956 <published>2020-10-21T04:19:23+00:00</published>
957 <updated>2020-10-21T04:19:23+00:00</updated>
958 <category term="/Release engineering" label="Release engineering" />
959 <category term="netbsd-9" scheme="http://roller.apache.org/ns/tags/" />
960 <category term="9.1" scheme="http://roller.apache.org/ns/tags/" />
961 <category term="release" scheme="http://roller.apache.org/ns/tags/" />
962 <category term="anouncement" scheme="http://roller.apache.org/ns/tags/" />
963 <summary type="html">&lt;p&gt;NetBSD 9.1, the first maintenance update for the NetBSD 9 branch, has been released&lt;/p&gt;</summary>
964 <content type="html">&lt;p&gt;After a small delay&lt;super&gt;&lt;a href=&quot;#footnote_delay&quot;&gt;*&lt;/a&gt;&lt;/super&gt;, the NetBSD Project is pleased to announce &lt;a href=&quot;https://mail-index.NetBSD.org/netbsd-announce/2020/10/21/msg000321.html&quot;&gt;NetBSD 9.1&lt;/a&gt;, the first feature and stability maintenance release of the netbsd-9 stable branch.&lt;/p&gt;
965 &lt;p&gt;
966 The new release features (among various other changes) many bug fixes,
967 a few performance enhancements, stability improvements for ZFS and LFS
968 and support for USB security keys in a mode easily usable in Firefox
969 and other applications.&lt;/p&gt;
970 &lt;p&gt;
971 For more details and instructions see the &lt;a href=&quot;https://www.netbsd.org/releases/formal-9/NetBSD-9.1.html&quot;&gt;9.1 announcement&lt;/a&gt;.&lt;/p&gt;
972 &lt;p&gt;
973 Get NetBSD 9.1 from our &lt;a href=&quot;https://cdn.netbsd.org/pub/NetBSD/NetBSD-9.1/&quot;&gt;CDN&lt;/a&gt; (provided by &lt;a href=&quot;http://www.fastly.com/&quot;&gt;fastly&lt;/a&gt;) or one of the ftp mirrors.&lt;/p&gt;
974 &lt;p&gt;
975 Complete source and binaries for NetBSD are available for download at many sites around the world. A list of download sites providing FTP, AnonCVS, and other services may be found at &lt;a href=&quot;https://www.NetBSD.org/mirrors/&quot;&gt;https://www.NetBSD.org/mirrors/&lt;/a&gt;.&lt;/p&gt;
976
977 &lt;p style=&quot;font-size: 0.8em&quot; name=&quot;footnote_delay&quot;&gt;* for the delay: let us say there was a minor hickup and we took the opportunity to provide up to date timezone files for NetBSD users in Fiji.&lt;/p&gt;</content>
978 </entry>
979 <entry>
980 <id>https://blog.netbsd.org/tnf/entry/google_summer_of_code_20202</id>
981 <title type="html">Google Summer of Code 2020: [Final Report] Enhancing Syzkaller support for NetBSD</title>
982 <author><name>Kamil Rytarowski</name></author>
983 <link rel="alternate" type="text/html" href="https://blog.netbsd.org/tnf/entry/google_summer_of_code_20202"/>
984 <published>2020-10-19T13:20:28+00:00</published>
985 <updated>2020-10-19T13:20:28+00:00</updated>
986 <category term="/The NetBSD Foundation" label="The NetBSD Foundation" />
987 <category term="syzkaller" scheme="http://roller.apache.org/ns/tags/" />
988 <summary type="html">This report was written by Ayushu Sharma as part of Google Summer of Code 2020.
989
990 &lt;p&gt;This post is a follow up of the &lt;a href=&quot;https://blog.netbsd.org/tnf/entry/gsoc_reports_enhancing_syzkaller_support&quot;&gt;first report&lt;/a&gt; and &lt;a href=&quot;https://blog.netbsd.org/tnf/entry/gsoc_reports_enhancing_syzkaller_support1&quot;&gt;second report&lt;/a&gt;. Post summarizes the work done during the third and final coding period for the Google Summer of Code (GSoc&#8217;20) project - &lt;a href=&quot;https://wiki.netbsd.org/projects/project/syzkaller/&quot;&gt;Enhance Syzkaller support for NetBSD&lt;/a&gt;&lt;/p&gt;</summary>
991 <content type="html">This report was written by Ayushu Sharma as part of Google Summer of Code 2020.
992
993 &lt;p&gt;This post is a follow up of the &lt;a href=&quot;https://blog.netbsd.org/tnf/entry/gsoc_reports_enhancing_syzkaller_support&quot;&gt;first report&lt;/a&gt; and &lt;a href=&quot;https://blog.netbsd.org/tnf/entry/gsoc_reports_enhancing_syzkaller_support1&quot;&gt;second report&lt;/a&gt;. Post summarizes the work done during the third and final coding period for the Google Summer of Code (GSoc&#8217;20) project - &lt;a href=&quot;https://wiki.netbsd.org/projects/project/syzkaller/&quot;&gt;Enhance Syzkaller support for NetBSD&lt;/a&gt;&lt;/p&gt;
994
995 &lt;h2&gt;Sys2syz&lt;/h2&gt;
996 &lt;p&gt;Sys2syz would give an extra edge to Syzkaller for NetBSD. It has a potential of efficiently automating the conversion of syscall definitions to syzkaller&#8217;s grammar. This can aid in increasing the number of syscalls covered by Syzkaller significantly with the minimum possibility of manual errors. Let&#8217;s delve into its internals.&lt;/p&gt;
997
998 &lt;h2&gt;A peek into Syz2syz Internals&lt;/h2&gt;
999
1000 &lt;p&gt;This tool parses the source code of device drivers present in C to a format which is compatible with grammar customized for syzkaller. Here, we try to cull the details of the target device by compiling, and then collocate the details with our python code. For further details about proposed design for the tool, refer to &lt;a href=&quot;https://blog.netbsd.org/tnf/entry/gsoc_reports_enhancing_syzkaller_support1&quot;&gt;previous post&lt;/a&gt;.&lt;p&gt;
1001
1002 &lt;p&gt;Python code follows 4 major steps:&lt;p&gt;
1003 &lt;ul&gt;
1004 &lt;li&gt;&lt;a href=&quot;https://github.com/ais2397/sys2syz/blob/master/core/Extractor.py&quot;&gt;&lt;b&gt;Extractor.py&lt;/b&gt;&lt;/a&gt; - Extraction of all ioctl commands of a given device driver along with their arguments from the header files.&lt;/li&gt;
1005 &lt;li&gt;&lt;a href=&quot;https://github.com/ais2397/sys2syz/blob/master/core/Bear.py&quot;&gt;&lt;b&gt;Bear.py&lt;/b&gt;&lt;/a&gt; - Preprocessing of the device driver&apos;s files using compile_commands.json generated during the setup of tool using Bear.&lt;/li&gt;
1006 &lt;li&gt;&lt;a href=&quot;https://github.com/ais2397/sys2syz/blob/master/core/C2xml.py&quot;&gt;&lt;b&gt;C2xml.py&lt;/b&gt;&lt;/a&gt; - XML files are generated by running c2xml on preprocessed device files. This eases the process of fetching the information related to arguments of commands&lt;/li&gt;
1007 &lt;li&gt;&lt;a href=&quot;https://github.com/ais2397/sys2syz/blob/master/core/Description.py&quot;&gt;&lt;b&gt;Description.py&lt;/b&gt;&lt;/a&gt; - Generates descriptions for the ioctl commands and their arguments (builtin-types, arrays, pointers, structures and unions) using the XML files.&lt;/li&gt;
1008 &lt;/ul&gt;
1009
1010 &lt;h3&gt;Extraction:&lt;/h3&gt;
1011 &lt;p&gt;This step involves fetching the possible ioctl commands for the target device driver and getting the files which have to be included in our dev_target.txt file. We have already seen all the commands for device drivers are defined in a specific way. These commands defined in the header files need to be grepped along with the major details, regex comes in as a rescue for this&lt;/p&gt;
1012
1013 &lt;pre&gt;
1014 &lt;code&gt;
1015 io = re.compile(&quot;#define\s+(.*)\s+_IO\((.*)\).*&quot;)
1016 iow = re.compile(&quot;#define\s+(.*)\s+_IOW\((.*),\s+(.*),\s+(.*)\).*&quot;)
1017 ior = re.compile(&quot;#define\s+(.*)\s+_IOR\((.*),\s+(.*),\s+(.*)\).*&quot;)
1018 iowr = re.compile(&quot;#define\s+(.*)\s+_IOWR\((.*),\s+(.*),\s+(.*)\).*&quot;)
1019 &lt;/code&gt;
1020 &lt;/pre&gt;
1021
1022 &lt;p&gt;
1023 Code scans through all the header files present in the target device folder and extracts all the commands along with their details using compiled regex expressions. Details include the direction of buffer(null, in, out, inout) based on the types of Ioctl calls(_IO, _IOR, _IOW, _IOWR) and the argument of the call. These are stored in a file named ioctl_commands.txt at location out/&amp;lttarget_name&amp;gt.
1024
1025 Example output:
1026 &lt;pre&gt;
1027 &lt;code&gt;
1028 out, I2C_IOCTL_EXEC, i2c_ioctl_exec_t
1029 &lt;/code&gt;
1030 &lt;/pre&gt;
1031
1032 &lt;h3&gt;Preprocessing:&lt;/h3&gt;
1033 &lt;p&gt;Preprocessing is required for getting XML files, about which we would look in the next step. Bear plays a major role when it comes to preprocessing C files. It records the commands executed for building the target device driver. This step is performed when setup.sh script is executed.&lt;/p&gt;
1034 &lt;p&gt;Extracted commands are modified with the help of parse_commands() function to include &#8216;-E&#8217; and &#8216;-fdirectives&#8217; flags and give it a new output location. Commands extracted by this function are then used by the compile_target function which filters out the unnecessary flags and generates preprocessed files in our output directory.&lt;/p&gt;
1035
1036 &lt;h3&gt;Generating XML files&lt;/h3&gt;
1037 &lt;p&gt;Run C2xml on the preprocessed files to fetch XML files which stores source code in a tree-like structure, making it easier to collect all the information related to each and every element of structures, unions etc. For eg: &lt;/p&gt;
1038 &lt;pre&gt;
1039 &lt;code&gt;
1040 &amp;ltsymbol type=&quot;struct&quot; id=&quot;_5970&quot; file=&quot;am2315.i&quot; start-line=&quot;13240&quot; start-col=&quot;16&quot; end-line=&quot;13244&quot; end-col=&quot;11&quot; bit-size=&quot;96&quot; alignment=&quot;4&quot; offset=&quot;0&quot;&amp;gt
1041 &amp;ltsymbol type=&quot;node&quot; id=&quot;_5971&quot; ident=&quot;ipending&quot; file=&quot;am2315.i&quot; start-line=&quot;13241&quot; start-col=&quot;33&quot; end-line=&quot;13241&quot; end-col=&quot;41&quot; bit-size=&quot;32&quot; alignment=&quot;4&quot; offset=&quot;0&quot; base-type-builtin=&quot;unsigned int&quot;/&amp;lt
1042 &amp;ltsymbol type=&quot;node&quot; id=&quot;_5972&quot; ident=&quot;ilevel&quot; file=&quot;am2315.i&quot; start-line=&quot;13242&quot; start-col=&quot;33&quot; end-line=&quot;13242&quot; end-col=&quot;39&quot; bit-size=&quot;32&quot; alignment=&quot;4&quot; offset=&quot;4&quot; base-type-builtin=&quot;int&quot;/&amp;gt
1043 &amp;ltsymbol type=&quot;node&quot; id=&quot;_5973&quot; ident=&quot;imasked&quot; file=&quot;am2315.i&quot; start-line=&quot;13243&quot; start-col=&quot;33&quot; end-line=&quot;13243&quot; end-col=&quot;40&quot; bit-size=&quot;32&quot; alignment=&quot;4&quot; offset=&quot;8&quot; base-type-builtin=&quot;unsigned int&quot;/&amp;gt
1044 &amp;lt/symbol&amp;gt
1045 &amp;ltsymbol type=&quot;pointer&quot; id=&quot;_5976&quot; file=&quot;am2315.i&quot; start-line=&quot;13249&quot; start-col=&quot;14&quot; end-line=&quot;13249&quot; end-col=&quot;25&quot; bit-size=&quot;64&quot; alignment=&quot;8&quot; offset=&quot;0&quot; base-type-builtin=&quot;void&quot;/&amp;gt
1046 &amp;ltsymbol type=&quot;array&quot; id=&quot;_5978&quot; file=&quot;am2315.i&quot; start-line=&quot;13250&quot; start-col=&quot;33&quot; end-line=&quot;13250&quot; end-col=&quot;39&quot; bit-size=&quot;288&quot; alignment=&quot;4&quot; offset=&quot;0&quot; base-type-builtin=&quot;unsigned int&quot; array-size=&quot;9&quot;/&amp;gt
1047 &lt;/code&gt;
1048 &lt;/pre&gt;
1049
1050 &lt;p&gt;We would further see how attributes like - idents, id, type, base-type-builtin etc conveniently helps us to analyze code and generate descriptions in a trouble-free manner .
1051 &lt;/p&gt;
1052
1053 &lt;h3&gt;Descriptions.py&lt;/h3&gt;
1054 &lt;p&gt;Final part, which offers a txt file storing all the required descriptions as its output. Here, information from the xml files and ioctl_commands.txt are combined together to generate descriptions of ioctl commands and their arguments.&lt;/p&gt;
1055 &lt;p&gt;Xml files for the given target device are parsed to form trees, &lt;/p&gt;
1056 &lt;pre&gt;
1057 &lt;code&gt;
1058 for file in (os.listdir(self.target)):
1059 tree = ET.parse(self.target+file)
1060 self.trees.append(tree)
1061 &lt;/code&gt;
1062 &lt;/pre&gt;
1063
1064 &lt;p&gt;We then traverse through these trees to search for the arguments of a particular ioctl command (particularly _IOR, _IOW, _IOWR commands) by the name of the argument. Once an element with the same value for ident attribute is found, attributes of the element are further examined to get its type. Possible types for these arguments are - struct, union, enum, function, array, pointer, macro and node. Using the type information we determine the way to define the element in accordance with syzkaller&#8217;s grammar syntax.
1065 &lt;/p&gt;
1066 &lt;p&gt;Building structs and unions involves defining their elements too, XML makes it easier. Program analyses each and every element which is a child of the root (struct/union) and generates its definitions. A dictionary helps in tracking the structs/unions which have been already built. Later, the dictionary is used to pretty print all the structs and union in the output file. Here is a code snippet which depicts the approach&lt;/p&gt;
1067
1068 &lt;pre&gt;
1069 &lt;code&gt;
1070 name = child.get(&quot;ident&quot;)
1071 if name not in self.structs_and_unions.keys():
1072 elements = {}
1073 for element in child:
1074 elem_type = self.get_type(element)
1075 elem_ident = element.get(&quot;ident&quot;)
1076 if elem_type == None:
1077 elem_type = element.get(&quot;type&quot;)
1078 elements[element.get(&quot;ident&quot;)] = elem_type
1079
1080 element_str = &quot;&quot;
1081 for element in elements:
1082 element_str += element + &quot;\t&quot; + elements[element] + &quot;\n&quot;
1083 self.structs_and_unions[name] = &quot; {\n&quot; + element_str + &quot;}\n&quot;
1084 return str(name)
1085 &lt;/code&gt;
1086 &lt;/pre&gt;
1087
1088 &lt;p&gt;Task of creating descriptions for arrays is made simpler due to the attribute - `array-size`.
1089 When it comes to dealing with pointers, syzkaller needs the user to fill in the direction of the pointer. This has already been taken care of while analyzing the ioctl commands in Extractor.py. The second argument with in/out/inout as its possible value depends on &#8216;fun&#8217; macros - _IOR, _IOW, _IOWR respectively. &lt;/p&gt;
1090
1091 &lt;p&gt;There is another category named as nodes which can be distinguished using the base-type-builtin and base-type attributes.
1092 &lt;/p&gt;
1093
1094 &lt;h2&gt;Result&lt;/h2&gt;
1095
1096 &lt;p&gt;Once the setup script for sys2syz is executed, sys2syz can be used for a certain target_device file by executing the python wrapper script (&lt;a href=&quot;https://github.com/ais2397/sys2syz/blob/master/sys2syz.py&quot;&gt;sys2syz.py&lt;/a&gt;) with :&lt;/p&gt;
1097 &lt;pre&gt;
1098 &lt;code&gt;#bin/sh
1099 python sys2syz.py -t &amp;ltabsolute_path_to_device_driver_source&amp;gt -c compile_commands.json -v
1100 &lt;/code&gt;
1101 &lt;/pre&gt;
1102 &lt;p&gt;This would generate a dev_&amp;ltdevice_driver&amp;gt.txt file in the out directory. An example description file autogenerated by sys2syz for i2c device driver.&lt;/p&gt;
1103
1104 &lt;pre&gt;
1105 &lt;code&gt;
1106 #Autogenerated by sys2syz
1107 include &lt;i2c_io.h&gt;
1108
1109 resource fd_i2c[fd]
1110
1111 syz_open_dev$I2C(dev ptr[in, string[&quot;/dev/i2c&quot;]], id intptr, flags flags[open_flags]) fd_i2c
1112
1113 ioctl$I2C_IOCTL_EXEC(fd fd_i2c, cmd const[I2C_IOCTL_EXEC], arg ptr[out, i2c_ioctl_exec])
1114
1115 i2c_ioctl_exec {
1116 iie_op flags[i2c_op_t_flags]
1117 iie_addr int16
1118 iie_buflen len[iie_buf, intptr]
1119 iie_buf buffer[out]
1120 iie_cmdlen len[iie_cmd, intptr]
1121 iie_cmd buffer[out]
1122 }
1123 &lt;/code&gt;
1124 &lt;/pre&gt;
1125
1126 &lt;h2&gt;Future Work&lt;/h2&gt;
1127 &lt;p&gt;Though we have a basic working structure of this tool, yet a lot has to be worked upon for leveling it up to make the best of it. Perfect goals would be met when there would be least of manual labor needed. Sys2syz still looks forward to automating the detection of macros used by the flag types in syzkaller. List of to-dos also includes extending syzkaller&#8217;s support for generation of description of syscalls.&lt;/p&gt;
1128 &lt;p&gt;Some other yet-to-be-done tasks include-
1129 &lt;ul&gt;
1130 &lt;li&gt; Generating descriptions for function type &lt;/li&gt;
1131 &lt;li&gt; Calculating attributes for structs and unions &lt;/li&gt;
1132 &lt;/ul&gt;
1133
1134 &lt;h2&gt;Summary&lt;/h2&gt;
1135
1136 &lt;p&gt;We have surely reached closer to our goals but the project needs active involvement and incremental updates to scale it up to its full potential. Looking forward to much more learning and making more contribution to NetBSD community.&lt;/p&gt;
1137 &lt;p&gt;Atlast, a word of thanks to my mentors William Coldwell, Siddharth Muralee, Santhosh Raju and Kamil Rytarowski as well as the NetBSD organization for being extremely supportive. Also, I owe a big thanks to Google for giving me such a glaring opportunity to work on this project.&lt;/p&gt;</content>
1138 </entry>
1139 <entry>
1140 <id>https://blog.netbsd.org/tnf/entry/the_gnu_gdb_debugger_and4</id>
1141 <title type="html">The GNU GDB Debugger and NetBSD (Part 5) </title>
1142 <author><name>Kamil Rytarowski</name></author>
1143 <link rel="alternate" type="text/html" href="https://blog.netbsd.org/tnf/entry/the_gnu_gdb_debugger_and4"/>
1144 <published>2020-10-07T17:16:53+00:00</published>
1145 <updated>2020-10-07T17:24:34+00:00</updated>
1146 <category term="/The NetBSD Foundation" label="The NetBSD Foundation" />
1147 <category term="binutils" scheme="http://roller.apache.org/ns/tags/" />
1148 <category term="gdbserver" scheme="http://roller.apache.org/ns/tags/" />
1149 <category term="gdb" scheme="http://roller.apache.org/ns/tags/" />
1150 <summary type="html">The NetBSD developers maintain two copies of GDB:
1151 &lt;ul&gt;
1152 &lt;li&gt;One in the base-system that includes a significant set of local patches.&lt;/li&gt;
1153 &lt;li&gt;Another one in pkgsrc whose patching is limited to mostly build fixes.&lt;/li&gt;
1154 &lt;/ul&gt;
1155 &lt;p&gt;
1156 The base-system version of GDB (GPLv3) still relies on local patching to work.
1157 I have set a goal to reduce the number of custom patches to bare minimum, ideally achieving the state of GDB working without any local modifications at all.</summary>
1158 <content type="html">The NetBSD developers maintain two copies of GDB:
1159 &lt;ul&gt;
1160 &lt;li&gt;One in the base-system that includes a significant set of local patches.&lt;/li&gt;
1161 &lt;li&gt;Another one in pkgsrc whose patching is limited to mostly build fixes.&lt;/li&gt;
1162 &lt;/ul&gt;
1163 &lt;p&gt;
1164 The base-system version of GDB (GPLv3) still relies on local patching to work.
1165 I have set a goal to reduce the number of custom patches to bare minimum, ideally achieving the state of GDB working without any local modifications at all.
1166 &lt;p&gt;
1167 &lt;h2&gt;GDB changes&lt;/h2&gt;
1168 &lt;p&gt;
1169 Last month, the NetBSD/amd64 support was merged into gdbserver.
1170 This month, the gdbserver target support was extended to
1171 &lt;a href=&quot;https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=8b667faedf6012048f1f6e71785b1ac1412b8a9c&quot;&gt;NetBSD/i386&lt;/a&gt; and
1172 &lt;a href=&quot;https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=8e1d09292902cff8325b08a64fa5a918c7f9aa4f&quot;&gt;NetBSD/aarch64&lt;/a&gt;.
1173 The gdbserver and gdb code was cleaned up, refactored and made capable of introducing even more NetBSD targets.
1174 &lt;p&gt;
1175 Meanwhile, the NetBSD/i386 build of GDB was
1176 &lt;a href=&quot;https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=1eb6eb795fd3479c97d8aadc4f70d6afad5f8511&quot;&gt;fixed&lt;/a&gt;.
1177 The missing include of x86-bsd-nat.h as a common header was added to i386-bsd-nat.h.
1178 The i386 GDB code for BSD contained a runtime assert that verified whether the locally hardcoded
1179 &lt;var&gt;struct sigcontext&lt;/var&gt; is compatible with the system headers. In reality, the system headers are no longer using
1180 this structure since 2003, after the switch to ucontext_t, and the validating code was no longer
1181 effective. After the switch to newer GCC, this was reported as a unused local variable by the compiler.
1182 I have decided to &lt;a href=&quot;https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=6ff330351e7741774c4b7ac1214cf7d73c7eac4d&quot;&gt;remove&lt;/a&gt; the check on NetBSD entirely.
1183 This was followed up by a small &lt;a href=&quot;https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=064280be25f2ff27477ce649f01a70d42ddad2ae&quot;&gt;build fix&lt;/a&gt;.
1184 &lt;p&gt;
1185 The NetBSD team has noticed that the GDB&apos;s agent.cc code contains a portability bug and prepared a local fix.
1186 The traditional behavior of the BSD kernel is that passing random values of sun_len (part of sockaddr_un)
1187 can cause failures. In order to prevent the problems, the sockaddr_un structure is now zeroed before use.
1188 I&apos;ve reimplemented the fix and successfully
1189 &lt;a href=&quot;https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=e2a2a24a8e78427ff8667d625f5befbe88c328bb&quot;&gt;upstreamed it&lt;/a&gt;.
1190 &lt;p&gt;
1191 In order to easily resolve the issue with environment hardening enforced by PaX MPROTECT,
1192 I&apos;ve &lt;a href=&quot;https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=91e5e8db334b9a87c54f03982dfa0c88e3c9d7a1&quot;&gt;introduced
1193 a runtime warning whenever byte transfers betweeen the debugee and debugger occur with the EACCES errno code&lt;/a&gt;.
1194 &lt;p&gt;
1195 &lt;h2&gt;binutils changes&lt;/h2&gt;
1196 &lt;p&gt;
1197 I&apos;ve added support for NetBSD/aarch64 upstream, in
1198 &lt;a href=&quot;https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=c0b313441717b65569edb01bf9984d2066d899de&quot;&gt;GNU BFD&lt;/a&gt; and
1199 &lt;a href=&quot;https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=cc8b27f89cc8d0fde4532134c19c40c47a023abd&quot;&gt;GNU GAS&lt;/a&gt;.
1200 NetBSD still carries local patches for the GNU binutils components, and GNU ld does not build out of the box on NetBSD/aarch64.
1201 &lt;p&gt;
1202 &lt;h2&gt;Summary&lt;/h2&gt;
1203 &lt;p&gt;
1204 The NetBSD support in GNU binutils and GDB is improving promptly, and the most popular
1205 platforms of amd64, i386 and aarch64 are getting proper support out of the box, without downstream patches.
1206 The remaining patches for these CPUs include: streamlining kgdb support,
1207 adding native GDB support for aarch64,
1208 upstreaming local modifications from the GNU binutils components (especially BFD and ld) and introducing portability enhancements
1209 in the dependent projects like libiberty and gnulib.
1210 Then, the remaining work is to streamline support for the remaining CPUs (Alpha, VAX, MIPS, HPPA, IA64, SH3, PPC, etc.),
1211 to develop the missing generic features (such as listing open file descriptors for the specified process) and
1212 to fix failures in the regression test-suite.</content>
1213 </entry>
1214 <entry>
1215 <id>https://blog.netbsd.org/tnf/entry/wayland_on_netbsd_trials_and</id>
1216 <title type="html">Wayland on NetBSD - trials and tribulations</title>
1217 <author><name>Nia Alarie</name></author>
1218 <link rel="alternate" type="text/html" href="https://blog.netbsd.org/tnf/entry/wayland_on_netbsd_trials_and"/>
1219 <published>2020-09-28T17:34:22+00:00</published>
1220 <updated>2020-09-28T17:34:22+00:00</updated>
1221 <category term="/Ports" label="Ports" />
1222 <summary type="html">&lt;p&gt;After I posted about the &lt;a href=&quot;https://blog.netbsd.org/tnf/entry/default_window_manager_switched_to&quot;&gt;new default window manager&lt;/a&gt; in NetBSD I got a few questions, including &quot;when is NetBSD switching from X11 to Wayland?&quot;, Wayland being X11&apos;s &amp;quot;new&amp;quot; rival. In this blog post, hopefully I can explain why we aren&apos;t yet!&lt;/p&gt;</summary>
1223 <content type="html">&lt;p&gt;After I posted about the &lt;a href=&quot;https://blog.netbsd.org/tnf/entry/default_window_manager_switched_to&quot;&gt;new default window manager&lt;/a&gt; in NetBSD I got a few questions, including &quot;when is NetBSD switching from X11 to Wayland?&quot;, Wayland being X11&apos;s &amp;quot;new&amp;quot; rival. In this blog post, hopefully I can explain why we aren&apos;t yet!&lt;/p&gt;
1224 &lt;p&gt;Last year (and early this year) I was responsible for porting the first working Wayland compositor to NetBSD - &lt;a href=&quot;https://github.com/michaelforney/swc&quot;&gt;swc&lt;/a&gt;. I chose it because it looked small and hackable. You can try it out by installing the &lt;code&gt;velox&lt;/code&gt; window manager from pkgsrc.&lt;/p&gt;
1225 &lt;a href=&quot;http://ftp.netbsd.org/pub/NetBSD/misc/nia/images/wayland-laptop.jpg&quot;&gt;&lt;img src=&quot;http://ftp.netbsd.org/pub/NetBSD/misc/nia/images/wayland-laptop.jpg&quot; style=&quot;max-width: 750px;&quot;
1226 alt=&quot;A Wayland compositor running on my NetBSD laptop, with a few applications like Luakit and Dungeon Crawl Stone Soup open.&quot;
1227 title=&quot;A Wayland compositor running on my NetBSD laptop, with a few applications like Luakit and Dungeon Crawl Stone Soup open.&quot;&gt;&lt;/a&gt;
1228 &lt;h2&gt;Difficulties&lt;/h2&gt;
1229 &lt;p&gt;In a Wayland system, the &amp;quot;compositor&amp;quot; (display server) is responsible for managing displays, input, and window management. Generally, this means a lot of OS-specific code is contained there.&lt;/p&gt;
1230 &lt;p&gt;Wayland does not define protocols for features X11 users expect, like screenshots, screen locking, or window management. Either you implement these inside the compositor (lots of work that has to be redone), or you define your own protocol extension.&lt;/p&gt;
1231 &lt;p&gt;The Wayland &amp;quot;reference implementation&amp;quot; is a small set of libraries that can be used to build a compositor or a client application. These libraries currently have hard dependencies on Linux kernel APIs like &lt;code&gt;epoll&lt;/code&gt;. In pkgsrc we&apos;ve patched the libraries to add &lt;a href=https://man.netbsd.org/kqueue.2&gt;kqueue(2)&lt;/a&gt; support, but the patches haven&apos;t been accepted upstream. Wayland is written with the assumption of Linux to the extent that every client application tends to &lt;code&gt;#include &amp;lt;linux/input.h&amp;gt;&lt;/code&gt; because Wayland&apos;s designers didn&apos;t see the need to define a OS-neutral way to get mouse button IDs.&lt;/p&gt;
1232 &lt;p&gt;So far, all Wayland compositors but swc have a hard dependency on libinput, which only supports Linux&apos;s input API (also cloned in FreeBSD). In NetBSD we have an entirely different input API - &lt;a href=&quot;https://man.netbsd.org/wscons.4&quot;&gt;wscons(4).&lt;/a&gt; wscons is actually fairly simple to write code for, someone just needs to go out there and do it. You can use my code in swc as a reference. :)&lt;/p&gt;
1233 &lt;p&gt;In general, Wayland is moving away from the modularity, portability, and standardization of the X server.&lt;/p&gt;
1234 &lt;h2&gt;Is it ready for production?&lt;/h2&gt;
1235 &lt;p&gt;No, but you can play with it.&lt;/p&gt;
1236 &lt;ul&gt;
1237 &lt;li&gt;swc has some remaining bugs and instability.&lt;/li&gt;
1238 &lt;li&gt;swc is incompatible with key applications like Firefox, but others like Luakit work, as do most things that use Qt5, GTK3, or SDL2. Not being able to run X11 applications currently is quite limiting.&lt;/li&gt;
1239 &lt;li&gt;Other popular compositors are not yet available. Alternatively, someone could write some new ones.&lt;/li&gt;
1240 &lt;li&gt;You need a supported GPU or SoC with kernel modesetting, since safe software fallbacks don&apos;t work here. So far, I&apos;ve only tested this with Intel GPUs.&lt;/li&gt;
1241 &lt;/ul&gt;
1242 &lt;h2&gt;Task list&lt;/h2&gt;
1243 &lt;ul&gt;
1244 &lt;li&gt;Adding support for wscons to more Wayland compositors and persuading developers to accept the patches.&lt;/li&gt;
1245 &lt;li&gt;Persuading developers not to add hard dependencies on &lt;code&gt;epoll&lt;/code&gt; and instead use an abstraction layer like libevent.&lt;/li&gt;
1246 &lt;li&gt;Updating the NetBSD kernel DRM/KMS stack. This is a difficult undertaking that involves porting code from the Linux kernel (a very fast moving target).
1247 &lt;ul&gt;
1248 &lt;li&gt;Getting support for newer DRM versions&lt;/li&gt;
1249 &lt;li&gt;Getting support for atomic modesetting&lt;/li&gt;
1250 &lt;li&gt;Getting support for Glamor X servers (for running X11 applications inside wayland, etc)&lt;/li&gt;
1251 &lt;li&gt;Newer AMDGPU drivers, etc&lt;/li&gt;
1252 &lt;/ul&gt;
1253 &lt;/li&gt;
1254 &lt;li&gt;Adding support for basic (non-DRMKMS) framebuffers to a Wayland compositor. X11 can run from a basic unaccelerated NetBSD framebuffer, but this isn&apos;t yet possible in any Wayland compositor.&lt;/li&gt;
1255 &lt;li&gt;Extending swc to add more features and fix bugs.&lt;/li&gt;
1256 &lt;/ul&gt;
1257 &lt;p&gt;I&apos;ve decided to take a break from this, since it&apos;s a fairly huge undertaking and uphill battle.
1258 Right now, X11 combined with a compositor like picom or xcompmgr is the more mature option.&lt;/p&gt;
1259 </content>
1260 </entry>
1261 <entry>
1262 <id>https://blog.netbsd.org/tnf/entry/default_window_manager_switched_to</id>
1263 <title type="html">Default window manager switched to CTWM in NetBSD-current</title>
1264 <author><name>Nia Alarie</name></author>
1265 <link rel="alternate" type="text/html" href="https://blog.netbsd.org/tnf/entry/default_window_manager_switched_to"/>
1266 <published>2020-09-28T08:33:28+00:00</published>
1267 <updated>2020-09-28T17:42:32+00:00</updated>
1268 <category term="/General" label="General" />
1269 <summary type="html">&lt;p&gt;
1270 For more than 20 years, NetBSD has shipped X11 with the &quot;classic&quot; default window manager of twm. However, it&apos;s been showing its age for a long time now.
1271 &lt;/p&gt;
1272 &lt;p&gt;
1273 In 2015, ctwm was imported, but after that no progress was made. ctwm is a fork of twm with some extra features - the primary advantages are that it&apos;s still incredibly lightweight, but highly configurable, and has support for virtual desktops, as well as a NetBSD-compatible license and ongoing development. Thanks to its configuration options, we can provide a default experience that&apos;s much more usable to people experienced with other operating systems.
1274 &lt;/p&gt;</summary>
1275 <content type="html">&lt;p&gt;
1276 For more than 20 years, NetBSD has shipped X11 with the &quot;classic&quot; default window manager of twm. However, it&apos;s been showing its age for a long time now.
1277 &lt;/p&gt;
1278 &lt;p&gt;
1279 In 2015, ctwm was imported, but after that no progress was made. ctwm is a fork of twm with some extra features - the primary advantages are that it&apos;s still incredibly lightweight, but highly configurable, and has support for virtual desktops, as well as a NetBSD-compatible license and ongoing development. Thanks to its configuration options, we can provide a default experience that&apos;s much more usable to people experienced with other operating systems.
1280 &lt;/p&gt;
1281 &lt;p&gt;
1282 Recently, I&apos;ve been installing NetBSD with some people in real life and was inspired by their reactions to the default twm to improve the situation, so I played with ctwm, wrote a config, and used it myself for a week. It&apos;s now the default in NetBSD-current.
1283 &lt;/p&gt;
1284 &lt;a href=&quot;https://ftp.netbsd.org/pub/NetBSD/misc/nia/images/desktop.png&quot;&gt;&lt;img src=&quot;https://ftp.netbsd.org/pub/NetBSD/misc/nia/images/desktop.png&quot; alt=&quot;&quot; style=&quot;max-width: 750px;&quot;&gt;&lt;/a&gt;
1285 &lt;p&gt;
1286 We gain some nice features like an auto-generated application menu (that will fill up as packages are installed to /usr/pkg), and a range of useful keyboard shortcuts including volume controls - the default config should be fully usable without a mouse. It should also work at a range of screen resolutions. We can add HiDPI support after some larger bitmap fonts are imported - another advantage of ctwm is that we can support very slow and very fast hardware with one config.
1287 &lt;/p&gt;
1288 &lt;p&gt;
1289 If you&apos;re curious about ctwm, check out the &lt;a href=&quot;https://www.ctwm.org/index.html&quot;&gt;ctwm website&lt;/a&gt;. It&apos;s also included in previous NetBSD releases, though not as the default window manager and not with this config.
1290 &lt;/p&gt;</content>
1291 </entry>
1292 <entry>
1293 <id>https://blog.netbsd.org/tnf/entry/google_summer_of_code_20201</id>
1294 <title type="html">Google Summer of Code 2020: [Final Report] RumpKernel Syscall Fuzzing</title>
1295 <author><name>Kamil Rytarowski</name></author>
1296 <link rel="alternate" type="text/html" href="https://blog.netbsd.org/tnf/entry/google_summer_of_code_20201"/>
1297 <published>2020-09-25T21:53:00+00:00</published>
1298 <updated>2020-10-19T13:22:39+00:00</updated>
1299 <category term="/The NetBSD Foundation" label="The NetBSD Foundation" />
1300 <category term="rump" scheme="http://roller.apache.org/ns/tags/" />
1301 <category term="gsoc" scheme="http://roller.apache.org/ns/tags/" />
1302 <category term="fuzzing" scheme="http://roller.apache.org/ns/tags/" />
1303 <content type="html">This report was prepared by Aditya Vardhan Padala as a part of Google Summer of Code 2020
1304
1305 &lt;p&gt;This post is the third update to the project RumpKernel Syscall Fuzzing.&lt;/p&gt;
1306 &lt;p&gt;Part1 - &lt;a href=&quot;https://blog.netbsd.org/tnf/entry/gsoc_reports_fuzzing_rumpkernel_syscalls1&quot;&gt;https://blog.netbsd.org/tnf/entry/gsoc_reports_fuzzing_rumpkernel_syscalls1&lt;/a&gt;&lt;/p&gt;
1307 &lt;p&gt;Part2 - &lt;a href=&quot;https://blog.netbsd.org/tnf/entry/gsoc_reports_fuzzing_rumpkernel_syscalls&quot;&gt;https://blog.netbsd.org/tnf/entry/gsoc_reports_fuzzing_rumpkernel_syscalls&lt;/a&gt;&lt;/p&gt;
1308 &lt;p&gt;The first and second coding period was entirely dedicated to fuzzing rumpkernel syscalls using hongfuzz. Initially a dumb fuzzer was developed to start fuzzing but it soon reached its limits.&lt;/p&gt;
1309 &lt;p&gt;For the duration of second coding peroid we concentrated on crash reproduction and adding grammar to the fuzzer which yielded in better results as we tested on a bug in ioctl with grammar. Although this works for now crash reproduction needs to be improved to generate a working c reproducer.&lt;/p&gt;
1310 &lt;p&gt;For the last coding period I have looked into the internals of syzkaller to understand how it pregenerates input and how it mutates data. I have continued to work on integrating &lt;a href=&quot;https://github.com/rumpkernel/buildrump.sh&quot;&gt;buildrump.sh&lt;/a&gt; with build.sh. buildrump eases the task fo building the rumpkernel on any host for any target.&lt;/p&gt;
1311 &lt;p&gt;&lt;a href=&quot;https://github.com/rumpkernel/buildrump.sh&quot;&gt;buildrump.sh&lt;/a&gt; is like a wrapper around build.sh to build the tools and rumpkernel from the source relevant to rumpkernel. So I worked to get buildrump.sh working with netbsd-src. Building the toolchain was successfull from netbsd-src. So binaries like rumpmake work just fine to continue building the rumpkernel.&lt;/p&gt;
1312 &lt;p&gt;But the rumpkernel failed to build due to some warnings and errors similar to the following. It can be due to the fact that buildrump.sh has been dormant recently I faced a lot of build issues.&lt;/p&gt;
1313 &lt;pre&gt;&lt;code&gt;&lt;span class=&quot;hljs-attribute&quot;&gt;nbmake[2]&lt;/span&gt;: nbmake[2]: don&apos;t know how to make /root/buildrump.sh/obj/dest.stage/usr/lib/crti.o. Stop
1314
1315 &lt;span class=&quot;crystal&quot;&gt;nbmake[&lt;span class=&quot;hljs-number&quot;&gt;2&lt;/span&gt;]: stopped in /root/buildrump.sh/src/&lt;span class=&quot;hljs-class&quot;&gt;&lt;span class=&quot;hljs-keyword&quot;&gt;lib&lt;/span&gt;/&lt;span class=&quot;hljs-title&quot;&gt;librumpuser&lt;/span&gt;&lt;/span&gt;
1316 &amp;gt;&amp;gt; &lt;span class=&quot;hljs-symbol&quot;&gt;ERROR:&lt;/span&gt;
1317 &amp;gt;&amp;gt; make /root/buildrump.sh/obj/Makefile.first dependall&lt;/span&gt;
1318 &lt;/code&gt;&lt;/pre&gt;&lt;p&gt;Few of the similar errors were easily fixed but I couldn&amp;#39;t integrate it during the time span of the coding period.&lt;/p&gt;
1319 &lt;p&gt;&lt;b&gt;To Do&lt;/b&gt;&lt;/p&gt;
1320 &lt;ul&gt;
1321 &lt;li&gt;Research more on grammar definition and look into the existing grammar fuzzers for a better understanding of generating grammar.&lt;/li&gt;
1322 &lt;li&gt;Integrate &lt;a href=&quot;https://github.com/ais2397/sys2syz&quot;&gt;syz2sys&lt;/a&gt; with the existing fuzzer to include grammar generation for better results.&lt;/li&gt;
1323 &lt;/ul&gt;
1324 &lt;p&gt;GSoC with NetBSD has been an amazing journey throughout, in which I had a chance to learn from awesome people and work on amazing projects. I will continue to work on the project to achieve the goal of integrating my fuzzer with OSS Fuzz. I thank my mentors Siddharth Muralee, Maciej Grochowski, Christos Zoulas for their support and Kamil for his continuous guidance.&lt;/p&gt;</content>
1325 </entry>
1326 <entry>
1327 <id>https://blog.netbsd.org/tnf/entry/google_summer_of_code_2020</id>
1328 <title type="html">Google Summer of Code 2020: [Final Report] Curses Library Automated Testing</title>
1329 <author><name>Kamil Rytarowski</name></author>
1330 <link rel="alternate" type="text/html" href="https://blog.netbsd.org/tnf/entry/google_summer_of_code_2020"/>
1331 <published>2020-09-25T21:50:01+00:00</published>
1332 <updated>2020-10-19T13:24:55+00:00</updated>
1333 <category term="/The NetBSD Foundation" label="The NetBSD Foundation" />
1334 <category term="curses" scheme="http://roller.apache.org/ns/tags/" />
1335 <summary type="html">This report was prepared by Naman Jain as a part of Google Summer of Code 2020
1336 &lt;p&gt;
1337 My GSoC project under NetBSD involves the development of the test framework of curses. This is the final blog report in a series of blog reports; you can look at the &lt;a href=&quot;https://blog.netbsd.org/tnf/entry/gsoc_reports_curses_library_automated&quot; rel=&quot;nofollow&quot;&gt;first report&lt;/a&gt; and &lt;a href=&quot;https://blog.netbsd.org/tnf/entry/gsoc_2020_second_evaluation_report&quot; rel=&quot;nofollow&quot;&gt;second report&lt;/a&gt; of the series.
1338 &lt;p&gt;The first report gives a brief introduction of the project and some insights into the curses testframe through its architecture and language. To someone who wants to contribute to the test suite, this blog can act as the quick guide of how things work internally. Meanwhile, the second report discusses some of the concepts that were quite challenging for me to understand. I wanted to share them with those who may face such a challenge. Both of these reports also cover the progress made in various phases of the Summer of Code.&lt;/p&gt;</summary>
1339 <content type="html">This report was prepared by Naman Jain as a part of Google Summer of Code 2020
1340 &lt;p&gt;
1341 My GSoC project under NetBSD involves the development of the test framework of curses. This is the final blog report in a series of blog reports; you can look at the &lt;a href=&quot;https://blog.netbsd.org/tnf/entry/gsoc_reports_curses_library_automated&quot; rel=&quot;nofollow&quot;&gt;first report&lt;/a&gt; and &lt;a href=&quot;https://blog.netbsd.org/tnf/entry/gsoc_2020_second_evaluation_report&quot; rel=&quot;nofollow&quot;&gt;second report&lt;/a&gt; of the series.
1342 &lt;p&gt;The first report gives a brief introduction of the project and some insights into the curses testframe through its architecture and language. To someone who wants to contribute to the test suite, this blog can act as the quick guide of how things work internally. Meanwhile, the second report discusses some of the concepts that were quite challenging for me to understand. I wanted to share them with those who may face such a challenge. Both of these reports also cover the progress made in various phases of the Summer of Code.&lt;/p&gt;
1343 &lt;p&gt;This being the final report in the series, I would love to share my experience throughout the project. I would be sharing some of the learning as well as caveats that I faced in the project.&lt;/p&gt;
1344 &lt;h2&gt;&lt;a id=&quot;user-content-challenges-and-caveats&quot; class=&quot;anchor&quot; aria-hidden=&quot;true&quot; href=&quot;#challenges-and-caveats&quot;&gt;&lt;svg class=&quot;octicon octicon-link&quot; viewBox=&quot;0 0 16 16&quot; version=&quot;1.1&quot; width=&quot;16&quot; height=&quot;16&quot; aria-hidden=&quot;true&quot;&gt;&lt;path fill-rule=&quot;evenodd&quot; d=&quot;M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z&quot;&gt;&lt;/path&gt;&lt;/svg&gt;&lt;/a&gt;Challenges and Caveats&lt;/h2&gt;
1345 &lt;p&gt;By the time my application for GSoC was submitted, I had gained some knowledge about the curses library and the testing framework. Combined with compiler design and library testing experience, that knowledge proved useful but not sufficient as I progressed through the project. There were times when, while writing a test case, you have to look into documentation from various sources, be it NetBSD, FreeBSD, Linux, Solaris, etc. One may find questioning his understanding of the framework, documentation, or even curses itself. This leads to the conclusion that for being a tester, one has to become a user first. That made me write minimal programs to understand the behavior. The experience was excellent, and I felt amazed by the capability and complexity of curses.&lt;/p&gt;
1346 &lt;h2&gt;&lt;a id=&quot;user-content-learnings&quot; class=&quot;anchor&quot; aria-hidden=&quot;true&quot; href=&quot;#learnings&quot;&gt;&lt;svg class=&quot;octicon octicon-link&quot; viewBox=&quot;0 0 16 16&quot; version=&quot;1.1&quot; width=&quot;16&quot; height=&quot;16&quot; aria-hidden=&quot;true&quot;&gt;&lt;path fill-rule=&quot;evenodd&quot; d=&quot;M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z&quot;&gt;&lt;/path&gt;&lt;/svg&gt;&lt;/a&gt;Learnings&lt;/h2&gt;
1347 &lt;p&gt;The foremost learning is from the experience of interacting with the open-source community and feeling confident in my abilities to contribute. Understanding the workflows; following the best practices like considering the maintainability, readability, and simplicity of the code were significant learning.&lt;/p&gt;
1348 &lt;p&gt;The project-specific learning was not limited to test-framework but a deeper understanding of curses as I have to browse through codes for the functions tested. As &lt;a href=&quot;https://www.linusakesson.net/programming/tty/&quot; rel=&quot;nofollow&quot;&gt;this&lt;/a&gt; blog says, getting the TTY demystified was a long-time desire, which got fulfilled to some extent.&lt;/p&gt;
1349 &lt;h2&gt;&lt;a id=&quot;user-content-some-tests-from-test-suite&quot; class=&quot;anchor&quot; aria-hidden=&quot;true&quot; href=&quot;#some-tests-from-test-suite&quot;&gt;&lt;svg class=&quot;octicon octicon-link&quot; viewBox=&quot;0 0 16 16&quot; version=&quot;1.1&quot; width=&quot;16&quot; height=&quot;16&quot; aria-hidden=&quot;true&quot;&gt;&lt;path fill-rule=&quot;evenodd&quot; d=&quot;M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z&quot;&gt;&lt;/path&gt;&lt;/svg&gt;&lt;/a&gt;Some tests from test suite&lt;/h2&gt;
1350 &lt;p&gt;In this section, I would discuss a couple of tests of the test suite written during the third phase of GSoC. Curses input model provides a variety of ways to obtain input from keyboard. We will consider 2 tests &lt;code&gt;keypad&lt;/code&gt; and &lt;code&gt;halfdelay&lt;/code&gt; that belong to input processing category but are somewhat unrelated.&lt;/p&gt;
1351 &lt;h3&gt;&lt;a id=&quot;user-content-keypad-processing&quot; class=&quot;anchor&quot; aria-hidden=&quot;true&quot; href=&quot;#keypad-processing&quot;&gt;&lt;svg class=&quot;octicon octicon-link&quot; viewBox=&quot;0 0 16 16&quot; version=&quot;1.1&quot; width=&quot;16&quot; height=&quot;16&quot; aria-hidden=&quot;true&quot;&gt;&lt;path fill-rule=&quot;evenodd&quot; d=&quot;M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z&quot;&gt;&lt;/path&gt;&lt;/svg&gt;&lt;/a&gt;Keypad Processing&lt;/h3&gt;
1352 &lt;p&gt;An application can enable or disable the tarnslation of keypad using &lt;code&gt;keypad()&lt;/code&gt; function. When translation is enabled, curses attempts to translate input sequence into a single key code. If disabled, curses passes the input as it is and any interpretation has to be made by application.&lt;/p&gt;
1353 &lt;pre&gt;&lt;code&gt;include window
1354 call $FALSE is_keypad $win1
1355 input &quot;\eOA&quot;
1356 call 0x1b wgetch $win1
1357 call OK keypad $win1 $TRUE
1358 input &quot;\eOA&quot;
1359 call $KEY_UP wgetch $win1
1360
1361 # disable assembly of KEY_UP
1362 call OK keyok $KEY_UP $FALSE
1363 input &quot;\eOA&quot;
1364 call 0x1b wgetch $win1
1365 &lt;/code&gt;&lt;/pre&gt;
1366 &lt;p&gt;As keypad translation is disabled by default, on input of &apos;\eOA&apos;, the input sequence is passed as it is and only &apos;\e&apos; (0x1b is hex code) is received on &lt;code&gt;wgetch()&lt;/code&gt;. If we enable the translation, then the same input is translated as KEY_UP. In curses, one can disable assembly of specific key symbols using &lt;code&gt;keyok()&lt;/code&gt;. See related &lt;a href=&quot;http://man.netbsd.org/keypad.3&quot; rel=&quot;nofollow&quot;&gt;man page&lt;/a&gt;.&lt;/p&gt;
1367 &lt;h3&gt;&lt;a id=&quot;user-content-input-mode&quot; class=&quot;anchor&quot; aria-hidden=&quot;true&quot; href=&quot;#input-mode&quot;&gt;&lt;svg class=&quot;octicon octicon-link&quot; viewBox=&quot;0 0 16 16&quot; version=&quot;1.1&quot; width=&quot;16&quot; height=&quot;16&quot; aria-hidden=&quot;true&quot;&gt;&lt;path fill-rule=&quot;evenodd&quot; d=&quot;M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z&quot;&gt;&lt;/path&gt;&lt;/svg&gt;&lt;/a&gt;Input Mode&lt;/h3&gt;
1368 &lt;p&gt;Curses lets the application control the effect of input using four input modes; cooked, cbreak, half-delay, raw. They specify the effect of input in terms of echo-ing and delay. We will discuss about the &lt;code&gt;halfdelay&lt;/code&gt; mode. The half-delay mode specifies how quickly certain curses function return to application when there is no terminal input waiting since the function is called.&lt;/p&gt;
1369 &lt;pre&gt;&lt;code&gt;include start
1370 delay 1000
1371 # input delay 1000 equals to 10 tenths of seconds
1372 # getch must fail for halfdelay(5) and pass for halfdelay(15)
1373 input &quot;a&quot;
1374 call OK halfdelay 15
1375 call 0x61 getch
1376 call OK halfdelay 5
1377 input &quot;a&quot;
1378 call -1 getch
1379 &lt;/code&gt;&lt;/pre&gt;
1380 &lt;p&gt;We have set the delay for feeding input to terminal with delay of 1s(10 tenths of second). If the application sets the halfdelay to 15, and makes a call to &lt;code&gt;getch()&lt;/code&gt; it receives the input. But it fails to get the input with haldelay set to 5. See related &lt;a href=&quot;http://man.netbsd.org/halfdelay.3&quot; rel=&quot;nofollow&quot;&gt;man page&lt;/a&gt;.&lt;/p&gt;
1381 &lt;h2&gt;&lt;a id=&quot;user-content-project-work&quot; class=&quot;anchor&quot; aria-hidden=&quot;true&quot; href=&quot;#project-work&quot;&gt;&lt;svg class=&quot;octicon octicon-link&quot; viewBox=&quot;0 0 16 16&quot; version=&quot;1.1&quot; width=&quot;16&quot; height=&quot;16&quot; aria-hidden=&quot;true&quot;&gt;&lt;path fill-rule=&quot;evenodd&quot; d=&quot;M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z&quot;&gt;&lt;/path&gt;&lt;/svg&gt;&lt;/a&gt;Project Work&lt;/h2&gt;
1382 &lt;p&gt;The &lt;a href=&quot;https://github.com/NamanJain8/curses&quot;&gt;work&lt;/a&gt; can be merged into organisation repository &lt;a href=&quot;https://github.com/NetBSD/src&quot;&gt;https://github.com/NetBSD/src&lt;/a&gt; under &lt;a href=&quot;https://github.com/NetBSD/src/tree/trunk/tests/lib/libcurses&quot;&gt;tests/lib/libcurses&lt;/a&gt;.&lt;/p&gt;
1383 &lt;p&gt;This project involved:&lt;/p&gt;
1384 &lt;ol&gt;
1385 &lt;li&gt;Improvement in testframework:
1386 &lt;ul&gt;
1387 &lt;li&gt;Automation of the checkfile generation.&lt;/li&gt;
1388 &lt;li&gt;Enhnacement of support for complex character&lt;/li&gt;
1389 &lt;li&gt;Addition of small features and code refactoring&lt;/li&gt;
1390 &lt;/ul&gt;
1391 &lt;/li&gt;
1392 &lt;li&gt;Testing and bug reports:
1393 &lt;ul&gt;
1394 &lt;li&gt;Tests for a family of routines like wide character, complex character, line drawing, box drawing, pad, window operations, cursor manipulations, soft label keys, input-output stream, and the ones involving their interactions.&lt;/li&gt;
1395 &lt;li&gt;Raising a bunch of Problem Report (PR) under &lt;code&gt;lib&lt;/code&gt; category some of which have been fixed. The list of PRs raised can be found &lt;a href=&quot;https://github.com/NamanJain8/curses/blob/master/reports/problem-reports.md&quot;&gt;here&lt;/a&gt;&lt;/li&gt;
1396 &lt;/ul&gt;
1397 &lt;/li&gt;
1398 &lt;/ol&gt;
1399 &lt;h2&gt;&lt;a id=&quot;user-content-future-work&quot; class=&quot;anchor&quot; aria-hidden=&quot;true&quot; href=&quot;#future-work&quot;&gt;&lt;svg class=&quot;octicon octicon-link&quot; viewBox=&quot;0 0 16 16&quot; version=&quot;1.1&quot; width=&quot;16&quot; height=&quot;16&quot; aria-hidden=&quot;true&quot;&gt;&lt;path fill-rule=&quot;evenodd&quot; d=&quot;M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z&quot;&gt;&lt;/path&gt;&lt;/svg&gt;&lt;/a&gt;Future Work&lt;/h2&gt;
1400 &lt;ul&gt;
1401 &lt;li&gt;The current testframe supports complex character, but the support needs to be extended for its string. This will enable testing of &lt;code&gt;[mv][w]add_wch[n]str&lt;/code&gt;, &lt;code&gt;[mv][w]in_wchstr&lt;/code&gt; family of routines.&lt;/li&gt;
1402 &lt;li&gt;Some of the tests for teminal manipulation routines like &lt;code&gt;intrflush&lt;/code&gt;, &lt;code&gt;def_prog_mode&lt;/code&gt;, &lt;code&gt;typeahead&lt;/code&gt;, &lt;code&gt;raw&lt;/code&gt;, etc. are not there in test suite.&lt;/li&gt;
1403 &lt;li&gt;Not specifically related to the framework, but the documentation for wide character as well as complex character routines need to be added.&lt;/li&gt;
1404 &lt;/ul&gt;
1405 &lt;h2&gt;&lt;a id=&quot;user-content-acknowledgements&quot; class=&quot;anchor&quot; aria-hidden=&quot;true&quot; href=&quot;#acknowledgements&quot;&gt;&lt;svg class=&quot;octicon octicon-link&quot; viewBox=&quot;0 0 16 16&quot; version=&quot;1.1&quot; width=&quot;16&quot; height=&quot;16&quot; aria-hidden=&quot;true&quot;&gt;&lt;path fill-rule=&quot;evenodd&quot; d=&quot;M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z&quot;&gt;&lt;/path&gt;&lt;/svg&gt;&lt;/a&gt;Acknowledgements&lt;/h2&gt;
1406 &lt;p&gt;I want to extend my heartfelt gratitude to my mentor Mr. Brett Lymn, who helped me through all the technical difficulties and challenges I faced. I also thank my mentor Martin Huseman for valuable suggestions and guidance at various junctures of the project. A special thanks to Kamil Rytarowski for making my blogs published on the NetBSD site.&lt;/p&gt;</content>
1407 </entry>
1408 <entry>
1409 <id>https://blog.netbsd.org/tnf/entry/gsoc_reports_benchmarking_netbsd_third</id>
1410 <title type="html">GSoC Reports: Benchmarking NetBSD, third evaluation report</title>
1411 <author><name>Leonardo Taccari</name></author>
1412 <link rel="alternate" type="text/html" href="https://blog.netbsd.org/tnf/entry/gsoc_reports_benchmarking_netbsd_third"/>
1413 <published>2020-09-12T08:29:06+00:00</published>
1414 <updated>2020-09-12T08:29:06+00:00</updated>
1415 <category term="/General" label="General" />
1416 <category term="gsoc2020" scheme="http://roller.apache.org/ns/tags/" />
1417 <category term="gsoc" scheme="http://roller.apache.org/ns/tags/" />
1418 <category term="benchmarks" scheme="http://roller.apache.org/ns/tags/" />
1419 <category term="pkgsrc" scheme="http://roller.apache.org/ns/tags/" />
1420 <summary type="html">&lt;p&gt;This report was written by Apurva Nandan as part of Google Summer of Code 2020.&lt;/p&gt;
1421
1422 &lt;p&gt;This blog post is in continuation of
1423 &lt;a href=&quot;//blog.NetBSD.org/tnf/entry/gsoc_reports_benchmarking_netbsd_first&quot;&gt;GSoC Reports: Benchmarking NetBSD, first evaluation report&lt;/a&gt;
1424 and &lt;a href=&quot;//blog.NetBSD.org/tnf/entry/gsoc_reports_benchmarking_netbsd_second&quot;&gt;GSoC Reports: Benchmarking NetBSD, second evaluation report&lt;/a&gt;
1425 blogs, and describes my progress in the final phase of GSoC 2020 under
1426 The NetBSD Foundation.&lt;/p&gt;
1427 &lt;p&gt;In the third phase, I upgraded to the latest stable version
1428 &lt;a href=&quot;https://www.phoronix-test-suite.com/&quot;&gt;Phoronix Test Suite (PTS)&lt;/a&gt; 9.8.0 in
1429 pkgsrc-wip, resolved the TODOs and created patches for more
1430 test-profiles to fix their installation and runtime errors on
1431 NetBSD-current.&lt;/p&gt;</summary>
1432 <content type="html">&lt;p&gt;This report was written by Apurva Nandan as part of Google Summer of Code 2020.&lt;/p&gt;
1433
1434 &lt;h2&gt;Introduction&lt;/h2&gt;
1435 &lt;p&gt;This blog post is in continuation of
1436 &lt;a href=&quot;//blog.NetBSD.org/tnf/entry/gsoc_reports_benchmarking_netbsd_first&quot;&gt;GSoC Reports: Benchmarking NetBSD, first evaluation report&lt;/a&gt;
1437 and &lt;a href=&quot;//blog.NetBSD.org/tnf/entry/gsoc_reports_benchmarking_netbsd_second&quot;&gt;GSoC Reports: Benchmarking NetBSD, second evaluation report&lt;/a&gt;
1438 blogs, and describes my progress in the final phase of GSoC 2020 under
1439 The NetBSD Foundation.&lt;/p&gt;
1440 &lt;p&gt;In the third phase, I upgraded to the latest stable version
1441 &lt;a href=&quot;https://www.phoronix-test-suite.com/&quot;&gt;Phoronix Test Suite (PTS)&lt;/a&gt; 9.8.0 in
1442 pkgsrc-wip, resolved the TODOs and created patches for more
1443 test-profiles to fix their installation and runtime errors on
1444 NetBSD-current.&lt;/p&gt;
1445
1446 &lt;h2&gt;Progress in the third phase of GSoC&lt;/h2&gt;
1447 &lt;h3&gt;wip/phoronix-test-suite TODO and update&lt;/h3&gt;
1448 &lt;p&gt;As a newer stable version of the Phoronix Test Suite was available in
1449 upstream, I upgraded the Phoronix Test Suite from version 9.6.1 to
1450 9.8.0 in pkgsrc-wip and is available as
1451 &lt;a href=&quot;https://pkgsrc.se/wip/phoronix-test-suite&quot;&gt;wip/phoronix-test-suite&lt;/a&gt;.
1452 You can have a look at the
1453 &lt;a href=&quot;https://github.com/phoronix-test-suite/phoronix-test-suite/blob/master/ChangeLog&quot;&gt;PTS Changelog&lt;/a&gt;
1454 to know about the improvements between these two versions.&lt;/p&gt;
1455 &lt;p&gt;To get the package ready for merge in pkgsrc upstream, I also resolved
1456 the pkgsrc-wip TODOs.&lt;/p&gt;
1457 &lt;h4&gt;pkgsrc-wip commits:&lt;/h4&gt;
1458 &lt;ul&gt;
1459 &lt;li&gt;&lt;a href=&quot;https://github.com/NetBSD/pkgsrc-wip/commit/77ef21385452fe098abb12f7772ac21f0aeb7f86&quot;&gt;wip/phoronix-test-suite: update phoronix-test-suite to 9.8.0&lt;/a&gt;&lt;/li&gt;
1460 &lt;li&gt;&lt;a href=&quot;https://github.com/NetBSD/pkgsrc-wip/commit/7162cf86c3fced1702a4446d514b76640d7266f3&quot;&gt;Resolved the pending TODOs&lt;/a&gt;&lt;/li&gt;
1461 &lt;/ul&gt;
1462 &lt;p&gt;If any new problems are encountered, please document them in
1463 &lt;code&gt;wip/phoronix-test-suite/TODO&lt;/code&gt; file and/or contact me.&lt;/p&gt;
1464 &lt;h3&gt;Testing of automated benchmarking framework&lt;/h3&gt;
1465 &lt;p&gt;I had been assigned a remote testing machine having Intel 6138 dual
1466 processor, 40 cores, 80 threads, 192GB of RAM. I spent time reproducing
1467 my automated framework i.e., Phoromatic-Anita Integration on the
1468 machine. I was able to reproduce the integration framework working
1469 without networking configuration, but the network bridge needs to be
1470 setup on the remote machine and the integration script to be tested
1471 with it. I shall continue this task in the post-GSoC period.&lt;/p&gt;
1472 &lt;h4&gt;Benchmarking Results&lt;/h4&gt;
1473 &lt;p&gt;I also performed benchmarking of NetBSD-9 amd64 native installation by
1474 running 50 test-profiles on a remote machine assigned to me by mentors
1475 and uploaded the benchmark results to
1476 &lt;a href=&quot;https://openbenchmarking.org/&quot;&gt;OpenBenchmarking.org&lt;/a&gt; at:&lt;/p&gt;
1477 &lt;ul&gt;
1478 &lt;li&gt;&lt;a href=&quot;https://openbenchmarking.org/result/2008287-NE-3966268951&quot;&gt;https://openbenchmarking.org/result/2008287-NE-3966268951&lt;/a&gt;&lt;/li&gt;
1479 &lt;/ul&gt;
1480 &lt;h3&gt;Test-profile debugging&lt;/h3&gt;
1481 &lt;p&gt;I then continued the task of maintaining/porting test-profiles and
1482 fixed the following test-profiles:&lt;/p&gt;
1483 &lt;h4&gt;Timed FFmpeg Compilation&lt;/h4&gt;
1484 &lt;p&gt;This test times how long it takes to build FFmpeg.
1485 This test is part of &lt;code&gt;Processor Test&lt;/code&gt; category.&lt;/p&gt;
1486 &lt;h5&gt;Original Test-profile:&lt;/h5&gt;
1487 &lt;p&gt;&lt;a href=&quot;https://openbenchmarking.org/test/pts/build-ffmpeg&quot;&gt;https://openbenchmarking.org/test/pts/build-ffmpeg&lt;/a&gt;&lt;/p&gt;
1488 &lt;h5&gt;Patched Test-profile:&lt;/h5&gt;
1489 &lt;p&gt;&lt;a href=&quot;https://github.com/apurvanandan1997/pts-test-profiles-dev/tree/master/build-ffmpeg-1.0.1&quot;&gt;https://github.com/apurvanandan1997/pts-test-profiles-dev/tree/master/build-ffmpeg-1.0.1&lt;/a&gt;&lt;/p&gt;
1490 &lt;h6&gt;Commit:&lt;/h6&gt;
1491 &lt;ul&gt;
1492 &lt;li&gt;&lt;a href=&quot;https://github.com/apurvanandan1997/pts-test-profiles-dev/commit/e9765185aabc01240cdb3671fb61f4c7d47e1b7e&quot;&gt;Replace the make -&amp;gt; gmake for compatibility with NetBSD&lt;/a&gt;&lt;/li&gt;
1493 &lt;/ul&gt;
1494 &lt;h4&gt;Compile Bench&lt;/h4&gt;
1495 &lt;p&gt;Compilebench tries to age a filesystem by simulating some of the disk
1496 IO common in creating, compiling, patching, stating and reading kernel
1497 trees. It indirectly measures how well filesystems can maintain
1498 directory locality as the disk fills up and directories age.
1499 This test is part of &lt;code&gt;Disk Test&lt;/code&gt; category.&lt;/p&gt;
1500 &lt;h5&gt;Original Test-profile:&lt;/h5&gt;
1501 &lt;p&gt;&lt;a href=&quot;https://openbenchmarking.org/test/pts/compilebench&quot;&gt;https://openbenchmarking.org/test/pts/compilebench&lt;/a&gt;&lt;/p&gt;
1502 &lt;h5&gt;Patched Test-profile:&lt;/h5&gt;
1503 &lt;p&gt;&lt;a href=&quot;https://github.com/apurvanandan1997/pts-test-profiles-dev/tree/master/compilebench-1.0.2&quot;&gt;https://github.com/apurvanandan1997/pts-test-profiles-dev/tree/master/compilebench-1.0.2&lt;/a&gt;&lt;/p&gt;
1504 &lt;h6&gt;Commit:&lt;/h6&gt;
1505 &lt;ul&gt;
1506 &lt;li&gt;&lt;a href=&quot;https://github.com/apurvanandan1997/pts-test-profiles-dev/commit/528b1853a07c22d79d2d8483b3fe2aa951d3866f&quot;&gt;Replaced python2 with NetBSD style naming python2.7&lt;/a&gt;&lt;/li&gt;
1507 &lt;/ul&gt;
1508 &lt;h4&gt;Timed MAFFT Alignment&lt;/h4&gt;
1509 &lt;p&gt;This test performs an alignment of 100 pyruvate decarboxylase sequences.
1510 This test is part of &lt;code&gt;Processor Test&lt;/code&gt; category.&lt;/p&gt;
1511 &lt;h5&gt;Original Test-profile:&lt;/h5&gt;
1512 &lt;p&gt;&lt;a href=&quot;https://openbenchmarking.org/test/pts/mafft&quot;&gt;https://openbenchmarking.org/test/pts/mafft&lt;/a&gt;&lt;/p&gt;
1513 &lt;h5&gt;Patched Test-profile:&lt;/h5&gt;
1514 &lt;p&gt;&lt;a href=&quot;https://github.com/apurvanandan1997/pts-test-profiles-dev/tree/master/mafft-1.5.0&quot;&gt;https://github.com/apurvanandan1997/pts-test-profiles-dev/tree/master/mafft-1.5.0&lt;/a&gt;&lt;/p&gt;
1515 &lt;h6&gt;Commits:&lt;/h6&gt;
1516 &lt;ul&gt;
1517 &lt;li&gt;&lt;a href=&quot;https://github.com/apurvanandan1997/pts-test-profiles-dev/commit/2bb288c4d352693b42586e55caf68a847d5740f8&quot;&gt;Replaced the make -&amp;gt; gmake for compatibility with NetBSD&lt;/a&gt;&lt;/li&gt;
1518 &lt;li&gt;&lt;a href=&quot;https://github.com/apurvanandan1997/pts-test-profiles-patches/commit/d31fee7256d165acfd22e23e04590ec322df3620&quot;&gt;Patch for replacing /bin/bash interpreter with /usr/pkg/bin/bash&lt;/a&gt;&lt;/li&gt;
1519 &lt;/ul&gt;
1520 &lt;h2&gt;Future Plans&lt;/h2&gt;
1521 &lt;p&gt;This officially summarizes my GSoC project: Benchmark NetBSD, and my
1522 end goal of the project that is to integrate Phoronix Test Suite with
1523 NetBSD and Anita for automated benchmarking is complete and its
1524 deployment on benchmark.NetBSD.org will be continued to be worked on
1525 with the coordination of moderators and merging the wip of Phoronix
1526 Test Suite 9.8.0 will be done by the pkgsrc maintainers in next days.&lt;/p&gt;
1527 &lt;p&gt;I want to thank my mentors and the NetBSD community without whose
1528 constant support I wouldn&apos;t have achieved the goals.&lt;/p&gt;</content>
1529 </entry>
1530 <entry>
1531 <id>https://blog.netbsd.org/tnf/entry/the_gnu_gdb_debugger_and3</id>
1532 <title type="html">The GNU GDB Debugger and NetBSD (Part 4)</title>
1533 <author><name>Kamil Rytarowski</name></author>
1534 <link rel="alternate" type="text/html" href="https://blog.netbsd.org/tnf/entry/the_gnu_gdb_debugger_and3"/>
1535 <published>2020-09-10T21:13:01+00:00</published>
1536 <updated>2020-09-10T21:17:53+00:00</updated>
1537 <category term="/The NetBSD Foundation" label="The NetBSD Foundation" />
1538 <category term="gdbserver" scheme="http://roller.apache.org/ns/tags/" />
1539 <category term="gdb" scheme="http://roller.apache.org/ns/tags/" />
1540 <summary type="html">The NetBSD team of developers maintains two copies of GDB:
1541 &lt;ul&gt;
1542 &lt;li&gt;One in the base-system with a stack of local patches.&lt;/li&gt;
1543 &lt;li&gt;One in pkgsrc with mostly build fix patches.&lt;/li&gt;
1544 &lt;/ul&gt;
1545 &lt;p&gt;
1546 The base-system version of GDB (GPLv3) still relies on a set of local patches.
1547 I set a goal to reduce the local patches to bare minimum, ideally reaching no local modifications at all.</summary>
1548 <content type="html">The NetBSD team of developers maintains two copies of GDB:
1549 &lt;ul&gt;
1550 &lt;li&gt;One in the base-system with a stack of local patches.&lt;/li&gt;
1551 &lt;li&gt;One in pkgsrc with mostly build fix patches.&lt;/li&gt;
1552 &lt;/ul&gt;
1553 &lt;p&gt;
1554 The base-system version of GDB (GPLv3) still relies on a set of local patches.
1555 I set a goal to reduce the local patches to bare minimum, ideally reaching no local modifications at all.
1556 &lt;p&gt;
1557 &lt;h1&gt;GDB changes&lt;/h1&gt;
1558 &lt;p&gt;
1559 Over the past month I worked on gdbserver for NetBSD/amd64 and finally
1560 &lt;a href=&quot;https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=62ba50486f1146f0cfd33074fc127fe00a02e87e&quot;&gt;upstreamed&lt;/a&gt; it to the GDB mainline, just in time for GDB 10.
1561 &lt;p&gt;
1562 What is gdbserver? Let&apos;s quote the official
1563 &lt;a href=&quot;https://sourceware.org/gdb/onlinedocs/gdb/Server.html&quot;&gt;GDB documentation&lt;/a&gt;:
1564 &lt;p&gt;
1565 &lt;i&gt;&lt;quote&gt;
1566 gdbserver is a control program for Unix-like systems, which allows you to connect your program with a remote GDB via target remote or target extended-but without linking in the usual debugging stub.
1567 &lt;p&gt;
1568 gdbserver is not a complete replacement for the debugging stubs, because it requires essentially the same operating-system facilities that GDB itself does. In fact, a system that can run gdbserver to connect to a remote GDB could also run GDB locally! gdbserver is sometimes useful nevertheless, because it is a much smaller program than GDB itself. It is also easier to port than all of GDB, so you may be able to get started more quickly on a new system by using gdbserver. Finally, if you develop code for real-time systems, you may find that the tradeoffs involved in real-time operation make it more convenient to do as much development work as possible on another system, for example by cross-compiling. You can use gdbserver to make a similar choice for debugging.
1569 &lt;p&gt;
1570 GDB and gdbserver communicate via either a serial line or a TCP connection, using the standard GDB remote serial protocol. remote
1571 &lt;/quote&gt;&lt;/i&gt;
1572 &lt;p&gt;
1573 This illustrated that gdbserver is especially useful for debugging applications on embedded and thin devices, connected to
1574 a controlling computer equipped with full distribution sources, toolchain, debugging information etc.
1575 Eventually, this approach of gdb and gdbserver can replace the native gdb plugin entirely and spawn all connections
1576 debugging sessions using this protocol.
1577 This design decision was already introduced into LLDB, where remote process plugin is the only supported program
1578 on Linux, NetBSD and highly recommended for other kernels.
1579 &lt;p&gt;
1580 I&apos;ve picked amd64 as the first target as it&apos;s the easiest to develop and test.
1581 &lt;p&gt;
1582 An example debugging session looks like this:
1583 &lt;pre&gt;
1584 $ uname -rms
1585 NetBSD 9.99.72 amd64
1586 $ LC_ALL=C date
1587 Thu Sep 10 22:43:10 CEST 2020
1588 $ ./gdbserver/gdbserver --version
1589 GNU gdbserver (GDB) 10.0.50.20200910-git
1590 Copyright (C) 2020 Free Software Foundation, Inc.
1591 gdbserver is free software, covered by the GNU General Public License.
1592 This gdbserver was configured as &quot;x86_64-unknown-netbsd9.99&quot;
1593 $ ./gdbserver/gdbserver localhost:1234 /usr/bin/nslookup
1594 Process /usr/bin/nslookup created; pid = 26383
1595 Listening on port 1234
1596 &lt;/pre&gt;
1597 &lt;p&gt;
1598 Then on the other terminal:
1599 &lt;p&gt;
1600 &lt;pre&gt;
1601 $ ./gdb/gdb
1602 GNU gdb (GDB) 10.0.50.20200910-git
1603 Copyright (C) 2020 Free Software Foundation, Inc.
1604 License GPLv3+: GNU GPL version 3 or later &lt;http://gnu.org/licenses/gpl.html&gt;
1605 This is free software: you are free to change and redistribute it.
1606 There is NO WARRANTY, to the extent permitted by law.
1607 Type &quot;show copying&quot; and &quot;show warranty&quot; for details.
1608 This GDB was configured as &quot;x86_64-unknown-netbsd9.99&quot;.
1609 Type &quot;show configuration&quot; for configuration details.
1610 For bug reporting instructions, please see:
1611 &lt;https://www.gnu.org/software/gdb/bugs/&gt;.
1612 Find the GDB manual and other documentation resources online at:
1613 --Type &lt;RET&gt; for more, q to quit, c to continue without paging--
1614 &lt;http://www.gnu.org/software/gdb/documentation/&gt;.
1615
1616 For help, type &quot;help&quot;.
1617 Type &quot;apropos word&quot; to search for commands related to &quot;word&quot;.
1618 (gdb) target remote localhost:1234
1619 Remote debugging using localhost:1234
1620 Reading /usr/bin/nslookup from remote target...
1621 warning: File transfers from remote targets can be slow. Use &quot;set sysroot&quot; to access files locally instead.
1622 Reading /usr/bin/nslookup from remote target...
1623 Reading symbols from target:/usr/bin/nslookup...
1624 Reading /usr/bin/nslookup.debug from remote target...
1625 Reading /usr/bin/.debug/nslookup.debug from remote target...
1626 Reading /usr/libdata/debug//usr/bin/nslookup.debug from remote target...
1627 Reading /usr/libdata/debug//usr/bin/nslookup.debug from remote target...
1628 Reading symbols from target:/usr/libdata/debug//usr/bin/nslookup.debug...
1629 process 28353 is executing new program: /usr/bin/nslookup
1630 Reading /usr/bin/nslookup from remote target...
1631 Reading /usr/bin/nslookup from remote target...
1632 Reading /usr/bin/nslookup.debug from remote target...
1633 Reading /usr/bin/.debug/nslookup.debug from remote target...
1634 Reading /usr/libdata/debug//usr/bin/nslookup.debug from remote target...
1635 Reading /usr/libdata/debug//usr/bin/nslookup.debug from remote target...
1636 Reading /usr/libexec/ld.elf_so from remote target...
1637 Reading /usr/libexec/ld.elf_so from remote target...
1638 Reading /usr/libexec/ld.elf_so.debug from remote target...
1639 Reading /usr/libexec/.debug/ld.elf_so.debug from remote target...
1640 Reading /usr/libdata/debug//usr/libexec/ld.elf_so.debug from remote target...
1641 Reading /usr/libdata/debug//usr/libexec/ld.elf_so.debug from remote target...
1642 warning: Invalid remote reply: timeout [kamil: repeated multiple times...]
1643 Reading /usr/lib/libbind9.so.15 from remote target...
1644 Reading /usr/lib/libisccfg.so.15 from remote target...
1645 Reading /usr/lib/libdns.so.15 from remote target...
1646 Reading /usr/lib/libns.so.15 from remote target...
1647 Reading /usr/lib/libirs.so.15 from remote target...
1648 Reading /usr/lib/libisccc.so.15 from remote target...
1649 Reading /usr/lib/libisc.so.15 from remote target...
1650 Reading /usr/lib/libkvm.so.6 from remote target...
1651 Reading /usr/lib/libz.so.1 from remote target...
1652 Reading /usr/lib/libblocklist.so.0 from remote target...
1653 Reading /usr/lib/libpthread.so.1 from remote target...
1654 Reading /usr/lib/libpthread.so.1.4.debug from remote target...
1655 Reading /usr/lib/.debug/libpthread.so.1.4.debug from remote target...
1656 Reading /usr/libdata/debug//usr/lib/libpthread.so.1.4.debug from remote target...
1657 Reading /usr/libdata/debug//usr/lib/libpthread.so.1.4.debug from remote target...
1658 Reading /usr/lib/libgssapi.so.11 from remote target...
1659 Reading /usr/lib/libheimntlm.so.5 from remote target...
1660 Reading /usr/lib/libkrb5.so.27 from remote target...
1661 Reading /usr/lib/libcom_err.so.8 from remote target...
1662 Reading /usr/lib/libhx509.so.6 from remote target...
1663 Reading /usr/lib/libcrypto.so.14 from remote target...
1664 Reading /usr/lib/libasn1.so.10 from remote target...
1665 Reading /usr/lib/libwind.so.1 from remote target...
1666 Reading /usr/lib/libheimbase.so.2 from remote target...
1667 Reading /usr/lib/libroken.so.20 from remote target...
1668 Reading /usr/lib/libsqlite3.so.1 from remote target...
1669 Reading /usr/lib/libcrypt.so.1 from remote target...
1670 Reading /usr/lib/libutil.so.7 from remote target...
1671 Reading /usr/lib/libedit.so.3 from remote target...
1672 Reading /usr/lib/libterminfo.so.2 from remote target...
1673 Reading /usr/lib/libc.so.12 from remote target...
1674 Reading /usr/lib/libgcc_s.so.1 from remote target...
1675 Reading symbols from target:/usr/lib/libbind9.so.15...
1676 Reading /usr/lib/libbind9.so.15.0.debug from remote target...
1677 Reading /usr/lib/.debug/libbind9.so.15.0.debug from remote target...
1678 Reading /usr/libdata/debug//usr/lib/libbind9.so.15.0.debug from remote target...
1679 Reading /usr/libdata/debug//usr/lib/libbind9.so.15.0.debug from remote target...
1680 Reading symbols from target:/usr/libdata/debug//usr/lib/libbind9.so.15.0.debug...
1681 Reading symbols from target:/usr/lib/libisccfg.so.15...
1682 Reading /usr/lib/libisccfg.so.15.0.debug from remote target...
1683 Reading /usr/lib/.debug/libisccfg.so.15.0.debug from remote target...
1684 Reading /usr/libdata/debug//usr/lib/libisccfg.so.15.0.debug from remote target...
1685 Reading /usr/libdata/debug//usr/lib/libisccfg.so.15.0.debug from remote target...
1686 --Type &lt;RET&gt; for more, q to quit, c to continue without paging--
1687 Reading symbols from target:/usr/libdata/debug//usr/lib/libisccfg.so.15.0.debug...
1688 Reading symbols from target:/usr/lib/libdns.so.15...
1689 Reading /usr/lib/libdns.so.15.0.debug from remote target...
1690 Reading /usr/lib/.debug/libdns.so.15.0.debug from remote target...
1691 Reading /usr/libdata/debug//usr/lib/libdns.so.15.0.debug from remote target...
1692 Reading /usr/libdata/debug//usr/lib/libdns.so.15.0.debug from remote target...
1693 Reading symbols from target:/usr/libdata/debug//usr/lib/libdns.so.15.0.debug...
1694 Reading symbols from target:/usr/lib/libns.so.15...
1695 Reading /usr/lib/libns.so.15.0.debug from remote target...
1696 Reading /usr/lib/.debug/libns.so.15.0.debug from remote target...
1697 Reading /usr/libdata/debug//usr/lib/libns.so.15.0.debug from remote target...
1698 Reading /usr/libdata/debug//usr/lib/libns.so.15.0.debug from remote target...
1699 Reading symbols from target:/usr/libdata/debug//usr/lib/libns.so.15.0.debug...
1700 Reading symbols from target:/usr/lib/libirs.so.15...
1701 Reading /usr/lib/libirs.so.15.0.debug from remote target...
1702 Reading /usr/lib/.debug/libirs.so.15.0.debug from remote target...
1703 Reading /usr/libdata/debug//usr/lib/libirs.so.15.0.debug from remote target...
1704 Reading /usr/libdata/debug//usr/lib/libirs.so.15.0.debug from remote target...
1705 Reading symbols from target:/usr/libdata/debug//usr/lib/libirs.so.15.0.debug...
1706 Reading symbols from target:/usr/lib/libisccc.so.15...
1707 Reading /usr/lib/libisccc.so.15.0.debug from remote target...
1708 Reading /usr/lib/.debug/libisccc.so.15.0.debug from remote target...
1709 Reading /usr/libdata/debug//usr/lib/libisccc.so.15.0.debug from remote target...
1710 Reading /usr/libdata/debug//usr/lib/libisccc.so.15.0.debug from remote target...
1711 Reading symbols from target:/usr/libdata/debug//usr/lib/libisccc.so.15.0.debug...
1712 Reading symbols from target:/usr/lib/libisc.so.15...
1713 Reading /usr/lib/libisc.so.15.0.debug from remote target...
1714 Reading /usr/lib/.debug/libisc.so.15.0.debug from remote target...
1715 Reading /usr/libdata/debug//usr/lib/libisc.so.15.0.debug from remote target...
1716 Reading /usr/libdata/debug//usr/lib/libisc.so.15.0.debug from remote target...
1717 Reading symbols from target:/usr/libdata/debug//usr/lib/libisc.so.15.0.debug...
1718 Reading symbols from target:/usr/lib/libkvm.so.6...
1719 Reading /usr/lib/libkvm.so.6.0.debug from remote target...
1720 Reading /usr/lib/.debug/libkvm.so.6.0.debug from remote target...
1721 Reading /usr/libdata/debug//usr/lib/libkvm.so.6.0.debug from remote target...
1722 Reading /usr/libdata/debug//usr/lib/libkvm.so.6.0.debug from remote target...
1723 Reading symbols from target:/usr/libdata/debug//usr/lib/libkvm.so.6.0.debug...
1724 Reading symbols from target:/usr/lib/libz.so.1...
1725 Reading /usr/lib/libz.so.1.0.debug from remote target...
1726 Reading /usr/lib/.debug/libz.so.1.0.debug from remote target...
1727 Reading /usr/libdata/debug//usr/lib/libz.so.1.0.debug from remote target...
1728 Reading /usr/libdata/debug//usr/lib/libz.so.1.0.debug from remote target...
1729 Reading symbols from target:/usr/libdata/debug//usr/lib/libz.so.1.0.debug...
1730 Reading symbols from target:/usr/lib/libblocklist.so.0...
1731 Reading /usr/lib/libblocklist.so.0.0.debug from remote target...
1732 Reading /usr/lib/.debug/libblocklist.so.0.0.debug from remote target...
1733 Reading /usr/libdata/debug//usr/lib/libblocklist.so.0.0.debug from remote target...
1734 Reading /usr/libdata/debug//usr/lib/libblocklist.so.0.0.debug from remote target...
1735 Reading symbols from target:/usr/libdata/debug//usr/lib/libblocklist.so.0.0.debug...
1736 Reading symbols from target:/usr/lib/libgssapi.so.11...
1737 Reading /usr/lib/libgssapi.so.11.0.debug from remote target...
1738 Reading /usr/lib/.debug/libgssapi.so.11.0.debug from remote target...
1739 Reading /usr/libdata/debug//usr/lib/libgssapi.so.11.0.debug from remote target...
1740 Reading /usr/libdata/debug//usr/lib/libgssapi.so.11.0.debug from remote target...
1741 Reading symbols from target:/usr/libdata/debug//usr/lib/libgssapi.so.11.0.debug...
1742 Reading symbols from target:/usr/lib/libheimntlm.so.5...
1743 Reading /usr/lib/libheimntlm.so.5.0.debug from remote target...
1744 Reading /usr/lib/.debug/libheimntlm.so.5.0.debug from remote target...
1745 Reading /usr/libdata/debug//usr/lib/libheimntlm.so.5.0.debug from remote target...
1746 Reading /usr/libdata/debug//usr/lib/libheimntlm.so.5.0.debug from remote target...
1747 Reading symbols from target:/usr/libdata/debug//usr/lib/libheimntlm.so.5.0.debug...
1748 Reading symbols from target:/usr/lib/libkrb5.so.27...
1749 Reading /usr/lib/libkrb5.so.27.0.debug from remote target...
1750 Reading /usr/lib/.debug/libkrb5.so.27.0.debug from remote target...
1751 Reading /usr/libdata/debug//usr/lib/libkrb5.so.27.0.debug from remote target...
1752 Reading /usr/libdata/debug//usr/lib/libkrb5.so.27.0.debug from remote target...
1753 Reading symbols from target:/usr/libdata/debug//usr/lib/libkrb5.so.27.0.debug...
1754 Reading symbols from target:/usr/lib/libcom_err.so.8...
1755 Reading /usr/lib/libcom_err.so.8.0.debug from remote target...
1756 Reading /usr/lib/.debug/libcom_err.so.8.0.debug from remote target...
1757 Reading /usr/libdata/debug//usr/lib/libcom_err.so.8.0.debug from remote target...
1758 Reading /usr/libdata/debug//usr/lib/libcom_err.so.8.0.debug from remote target...
1759 Reading symbols from target:/usr/libdata/debug//usr/lib/libcom_err.so.8.0.debug...
1760 Reading symbols from target:/usr/lib/libhx509.so.6...
1761 Reading /usr/lib/libhx509.so.6.0.debug from remote target...
1762 Reading /usr/lib/.debug/libhx509.so.6.0.debug from remote target...
1763 Reading /usr/libdata/debug//usr/lib/libhx509.so.6.0.debug from remote target...
1764 Reading /usr/libdata/debug//usr/lib/libhx509.so.6.0.debug from remote target...
1765 Reading symbols from target:/usr/libdata/debug//usr/lib/libhx509.so.6.0.debug...
1766 Reading symbols from target:/usr/lib/libcrypto.so.14...
1767 Reading /usr/lib/libcrypto.so.14.0.debug from remote target...
1768 Reading /usr/lib/.debug/libcrypto.so.14.0.debug from remote target...
1769 Reading /usr/libdata/debug//usr/lib/libcrypto.so.14.0.debug from remote target...
1770 Reading /usr/libdata/debug//usr/lib/libcrypto.so.14.0.debug from remote target...
1771 Reading symbols from target:/usr/libdata/debug//usr/lib/libcrypto.so.14.0.debug...
1772 Reading symbols from target:/usr/lib/libasn1.so.10...
1773 Reading /usr/lib/libasn1.so.10.0.debug from remote target...
1774 Reading /usr/lib/.debug/libasn1.so.10.0.debug from remote target...
1775 Reading /usr/libdata/debug//usr/lib/libasn1.so.10.0.debug from remote target...
1776 Reading /usr/libdata/debug//usr/lib/libasn1.so.10.0.debug from remote target...
1777 Reading symbols from target:/usr/libdata/debug//usr/lib/libasn1.so.10.0.debug...
1778 Reading symbols from target:/usr/lib/libwind.so.1...
1779 Reading /usr/lib/libwind.so.1.0.debug from remote target...
1780 Reading /usr/lib/.debug/libwind.so.1.0.debug from remote target...
1781 Reading /usr/libdata/debug//usr/lib/libwind.so.1.0.debug from remote target...
1782 Reading /usr/libdata/debug//usr/lib/libwind.so.1.0.debug from remote target...
1783 Reading symbols from target:/usr/libdata/debug//usr/lib/libwind.so.1.0.debug...
1784 Reading symbols from target:/usr/lib/libheimbase.so.2...
1785 Reading /usr/lib/libheimbase.so.2.0.debug from remote target...
1786 Reading /usr/lib/.debug/libheimbase.so.2.0.debug from remote target...
1787 Reading /usr/libdata/debug//usr/lib/libheimbase.so.2.0.debug from remote target...
1788 Reading /usr/libdata/debug//usr/lib/libheimbase.so.2.0.debug from remote target...
1789 Reading symbols from target:/usr/libdata/debug//usr/lib/libheimbase.so.2.0.debug...
1790 Reading symbols from target:/usr/lib/libroken.so.20...
1791 Reading /usr/lib/libroken.so.20.0.debug from remote target...
1792 Reading /usr/lib/.debug/libroken.so.20.0.debug from remote target...
1793 Reading /usr/libdata/debug//usr/lib/libroken.so.20.0.debug from remote target...
1794 Reading /usr/libdata/debug//usr/lib/libroken.so.20.0.debug from remote target...
1795 Reading symbols from target:/usr/libdata/debug//usr/lib/libroken.so.20.0.debug...
1796 Reading symbols from target:/usr/lib/libsqlite3.so.1...
1797 Reading /usr/lib/libsqlite3.so.1.4.debug from remote target...
1798 Reading /usr/lib/.debug/libsqlite3.so.1.4.debug from remote target...
1799 Reading /usr/libdata/debug//usr/lib/libsqlite3.so.1.4.debug from remote target...
1800 Reading /usr/libdata/debug//usr/lib/libsqlite3.so.1.4.debug from remote target...
1801 Reading symbols from target:/usr/libdata/debug//usr/lib/libsqlite3.so.1.4.debug...
1802 Reading symbols from target:/usr/lib/libcrypt.so.1...
1803 Reading /usr/lib/libcrypt.so.1.0.debug from remote target...
1804 Reading /usr/lib/.debug/libcrypt.so.1.0.debug from remote target...
1805 Reading /usr/libdata/debug//usr/lib/libcrypt.so.1.0.debug from remote target...
1806 Reading /usr/libdata/debug//usr/lib/libcrypt.so.1.0.debug from remote target...
1807 Reading symbols from target:/usr/libdata/debug//usr/lib/libcrypt.so.1.0.debug...
1808 Reading symbols from target:/usr/lib/libutil.so.7...
1809 Reading /usr/lib/libutil.so.7.24.debug from remote target...
1810 Reading /usr/lib/.debug/libutil.so.7.24.debug from remote target...
1811 Reading /usr/libdata/debug//usr/lib/libutil.so.7.24.debug from remote target...
1812 Reading /usr/libdata/debug//usr/lib/libutil.so.7.24.debug from remote target...
1813 Reading symbols from target:/usr/libdata/debug//usr/lib/libutil.so.7.24.debug...
1814 Reading symbols from target:/usr/lib/libedit.so.3...
1815 Reading /usr/lib/libedit.so.3.1.debug from remote target...
1816 Reading /usr/lib/.debug/libedit.so.3.1.debug from remote target...
1817 Reading /usr/libdata/debug//usr/lib/libedit.so.3.1.debug from remote target...
1818 Reading /usr/libdata/debug//usr/lib/libedit.so.3.1.debug from remote target...
1819 Reading symbols from target:/usr/libdata/debug//usr/lib/libedit.so.3.1.debug...
1820 Reading symbols from target:/usr/lib/libterminfo.so.2...
1821 Reading /usr/lib/libterminfo.so.2.0.debug from remote target...
1822 Reading /usr/lib/.debug/libterminfo.so.2.0.debug from remote target...
1823 Reading /usr/libdata/debug//usr/lib/libterminfo.so.2.0.debug from remote target...
1824 Reading /usr/libdata/debug//usr/lib/libterminfo.so.2.0.debug from remote target...
1825 Reading symbols from target:/usr/libdata/debug//usr/lib/libterminfo.so.2.0.debug...
1826 Reading symbols from target:/usr/lib/libc.so.12...
1827 Reading /usr/lib/libc.so.12.217.debug from remote target...
1828 Reading /usr/lib/.debug/libc.so.12.217.debug from remote target...
1829 Reading /usr/libdata/debug//usr/lib/libc.so.12.217.debug from remote target...
1830 Reading /usr/libdata/debug//usr/lib/libc.so.12.217.debug from remote target...
1831 Reading symbols from target:/usr/libdata/debug//usr/lib/libc.so.12.217.debug...
1832 Reading symbols from target:/usr/lib/libgcc_s.so.1...
1833 Reading /usr/lib/libgcc_s.so.1.0.debug from remote target...
1834 Reading /usr/lib/.debug/libgcc_s.so.1.0.debug from remote target...
1835 Reading /usr/libdata/debug//usr/lib/libgcc_s.so.1.0.debug from remote target...
1836 Reading /usr/libdata/debug//usr/lib/libgcc_s.so.1.0.debug from remote target...
1837 Reading symbols from target:/usr/libdata/debug//usr/lib/libgcc_s.so.1.0.debug...
1838 Reading /usr/libexec/ld.elf_so from remote target...
1839 _rtld_debug_state () at /usr/src/libexec/ld.elf_so/rtld.c:1577
1840 1577 __insn_barrier();
1841 (gdb) b main
1842 Breakpoint 1 at 0x211c00: file /usr/src/external/mpl/bind/bin/nslookup/../../dist/bin/dig/nslookup.c, line 990.
1843 (gdb) c
1844 Continuing.
1845
1846 Breakpoint 1, main (argc=1, argv=0x7f7fffffe768)
1847 at /usr/src/external/mpl/bind/bin/nslookup/../../dist/bin/dig/nslookup.c:990
1848 990 main(int argc, char **argv) {
1849 (gdb) bt
1850 #0 main (argc=1, argv=0x7f7fffffe768)
1851 at /usr/src/external/mpl/bind/bin/nslookup/../../dist/bin/dig/nslookup.c:990
1852 (gdb) info threads
1853 Id Target Id Frame
1854 * 1 Thread 28353.28353 main (argc=1, argv=0x7f7fffffe768)
1855 at /usr/src/external/mpl/bind/bin/nslookup/../../dist/bin/dig/nslookup.c:990
1856 (gdb) b pthread_setname_np
1857 Breakpoint 2 at 0x7f7ff4e0c9e4: file /usr/src/lib/libpthread/pthread.c, line 792.
1858 (gdb) c
1859 Continuing.
1860 [New Thread 28353.27773]
1861
1862 Thread 1 hit Breakpoint 2, pthread_setname_np (thread=0x7f7ff7e41000,
1863 name=name@entry=0x7f7fffffe610 &quot;work-0&quot;, arg=arg@entry=0x0)
1864 at /usr/src/lib/libpthread/pthread.c:792
1865 792 {
1866 (gdb) info threads
1867 Id Target Id Frame
1868 * 1 Thread 28353.28353 pthread_setname_np (thread=0x7f7ff7e41000,
1869 name=name@entry=0x7f7fffffe610 &quot;work-0&quot;, arg=arg@entry=0x0)
1870 at /usr/src/lib/libpthread/pthread.c:792
1871 2 Thread 28353.27773 0x00007f7ff0aa623a in ___lwp_park60 () from target:/usr/lib/libc.so.12
1872 (gdb) n
1873 796 pthread__error(EINVAL, &quot;Invalid thread&quot;,
1874 (gdb) n
1875 799 if (pthread__find(thread) != 0)
1876 (gdb)
1877 802 namelen = snprintf(newname, sizeof(newname), name, arg);
1878 (gdb)
1879 803 if (namelen &gt;= PTHREAD_MAX_NAMELEN_NP)
1880 (gdb)
1881 806 cp = strdup(newname);
1882 (gdb)
1883 807 if (cp == NULL)
1884 (gdb)
1885 810 pthread_mutex_lock(&amp;thread-&gt;pt_lock);
1886 (gdb)
1887 811 oldname = thread-&gt;pt_name;
1888 (gdb)
1889 812 thread-&gt;pt_name = cp;
1890 (gdb)
1891 813 (void)_lwp_setname(thread-&gt;pt_lid, cp);
1892 (gdb)
1893 814 pthread_mutex_unlock(&amp;thread-&gt;pt_lock);
1894 (gdb) n
1895 816 if (oldname != NULL)
1896 (gdb) n
1897 isc_taskmgr_create (mctx=&lt;optimized out&gt;, workers=workers@entry=1, default_quantum=&lt;optimized out&gt;,
1898 default_quantum@entry=0, nm=nm@entry=0x0, managerp=managerp@entry=0x418638 &lt;taskmgr&gt;)
1899 at /usr/src/external/mpl/bind/lib/libisc/../../dist/lib/isc/task.c:1431
1900 1431 for (i = 0; i &lt; workers; i++) {
1901 (gdb) info threads
1902 Id Target Id Frame
1903 * 1 Thread 28353.28353 isc_taskmgr_create (mctx=&lt;optimized out&gt;, workers=workers@entry=1,
1904 default_quantum=&lt;optimized out&gt;, default_quantum@entry=0, nm=nm@entry=0x0,
1905 managerp=managerp@entry=0x418638 &lt;taskmgr&gt;)
1906 at /usr/src/external/mpl/bind/lib/libisc/../../dist/lib/isc/task.c:1431
1907 2 Thread 28353.27773 &quot;work-0&quot; 0x00007f7ff0aa623a in ___lwp_park60 ()
1908 from target:/usr/lib/libc.so.12
1909 (gdb) dis 1
1910 (gdb) b exit
1911 Breakpoint 3 at 0x7f7ff0b530e0: exit. (2 locations)
1912 (gdb) c
1913 Continuing.
1914
1915 Thread 1 hit Breakpoint 2, pthread_setname_np (thread=0x7f7ff7e42c00,
1916 name=name@entry=0x7f7ff5e6324e &quot;isc-timer&quot;, arg=arg@entry=0x0)
1917 at /usr/src/lib/libpthread/pthread.c:792
1918 792 {
1919 (gdb) dis 2
1920 (gdb) c
1921 Continuing.
1922 Reading /usr/lib/i18n/libUTF8.so.5.0 from remote target...
1923 Reading /usr/lib/i18n/libUTF8.so.5.0.debug from remote target...
1924 Reading /usr/lib/i18n/.debug/libUTF8.so.5.0.debug from remote target...
1925 Reading /usr/libdata/debug//usr/lib/i18n/libUTF8.so.5.0.debug from remote target...
1926 Reading /usr/libdata/debug//usr/lib/i18n/libUTF8.so.5.0.debug from remote target...
1927 &lt;/pre&gt;
1928 &lt;p&gt;
1929 Then, back to the first terminal:
1930 &lt;pre&gt;
1931 &gt; netbsd.org
1932 Server: 62.179.1.62
1933 Address: 62.179.1.62#53
1934
1935 Non-authoritative answer:
1936 Name: netbsd.org
1937 Address: 199.233.217.205
1938 Name: netbsd.org
1939 Address: 2001:470:a085:999::80
1940 &gt; exit
1941
1942 &lt;/pre&gt;
1943 &lt;p&gt;
1944 &lt;pre&gt;
1945 Thread 1 hit Breakpoint 3, exit (status=1) at /usr/src/lib/libc/stdlib/exit.c:55
1946 55 {
1947 (gdb) info threads
1948 Id Target Id Frame
1949 * 1 Thread 28353.28353 exit (status=1) at /usr/src/lib/libc/stdlib/exit.c:55
1950 (gdb) bt
1951 #0 exit (status=1) at /usr/src/lib/libc/stdlib/exit.c:55
1952 #1 0x0000000000206122 in ___start ()
1953 #2 0x00007f7ff7c0c840 in ?? () from target:/usr/libexec/ld.elf_so
1954 #3 0x0000000000000001 in ?? ()
1955 #4 0x00007f7fffffed20 in ?? ()
1956 #5 0x0000000000000000 in ?? ()
1957 (gdb) kill
1958 Kill the program being debugged? (y or n) y
1959 [Inferior 1 (process 28353) killed]
1960 &lt;/pre&gt;
1961 &lt;p&gt;
1962 &lt;b&gt;It worked!&lt;/b&gt;
1963 &lt;p&gt;
1964 In order to get this functionality operational I had to implement multiple GDB functions, in particular: create_inferior,
1965 post_create_inferior, attach, kill, detach, mourn, join, thread_alive,
1966 resume, wait, fetch_registers, store_registers, read_memory, write_memory,
1967 request_interrupt, supports_read_auxv, read_auxv,
1968 supports_hardware_single_step, sw_breakpoint_from_kind,
1969 supports_z_point_type, insert_point, remove_point,
1970 stopped_by_sw_breakpoint, supports_qxfer_siginfo, qxfer_siginfo,
1971 supports_stopped_by_sw_breakpoint, supports_non_stop,
1972 supports_multi_process, supports_fork_events, supports_vfork_events,
1973 supports_exec_events, supports_disable_randomization,
1974 supports_qxfer_libraries_svr4, qxfer_libraries_svr4,
1975 supports_pid_to_exec_file, pid_to_exec_file, thread_name,
1976 supports_catch_syscall.
1977 &lt;p&gt;
1978 NetBSD is the first BSD and actually the first Open Source UNIX-like OS besides Linux to grow support for gdbserver.
1979 &lt;p&gt;
1980 &lt;h1&gt;Plan for the next milestone&lt;/h1&gt;
1981 &lt;p&gt;
1982 Introduce AArch64 support for GDB/NetBSD.</content>
1983 </entry>
1984 <entry>
1985 <id>https://blog.netbsd.org/tnf/entry/gsoc_2020_report_2_fuzzing</id>
1986 <title type="html">GSoC 2020: Report-2: Fuzzing the NetBSD Network Stack in a Rumpkernel Environment</title>
1987 <author><name>Kamil Rytarowski</name></author>
1988 <link rel="alternate" type="text/html" href="https://blog.netbsd.org/tnf/entry/gsoc_2020_report_2_fuzzing"/>
1989 <published>2020-08-30T13:21:57+00:00</published>
1990 <updated>2020-08-30T13:21:57+00:00</updated>
1991 <category term="/The NetBSD Foundation" label="The NetBSD Foundation" />
1992 <category term="gsoc" scheme="http://roller.apache.org/ns/tags/" />
1993 <category term="rump" scheme="http://roller.apache.org/ns/tags/" />
1994 <category term="fuzzing" scheme="http://roller.apache.org/ns/tags/" />
1995 <summary type="html">This report was written by Nisarg S. Joshi as part of Google Summer of Code 2020.
1996
1997 &lt;p&gt;The objective of this project is to fuzz the various protocols and layers of the network stack of NetBSD using rumpkernel. This project is being carried out as a part of GSoC 2020. This blog post is regarding the project, the concepts and tools involved, the objectives and the current progress and next steps.&lt;/p&gt;
1998 &lt;p&gt;You can read the previous post/report &lt;a href=&quot;http://blog.netbsd.org/tnf/entry/gsoc_reports_fuzzing_the_netbsd&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;</summary>
1999 <content type="html">This report was written by Nisarg S. Joshi as part of Google Summer of Code 2020.
2000
2001 &lt;p&gt;The objective of this project is to fuzz the various protocols and layers of the network stack of NetBSD using rumpkernel. This project is being carried out as a part of GSoC 2020. This blog post is regarding the project, the concepts and tools involved, the objectives and the current progress and next steps.&lt;/p&gt;
2002 &lt;p&gt;You can read the previous post/report &lt;a href=&quot;http://blog.netbsd.org/tnf/entry/gsoc_reports_fuzzing_the_netbsd&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;
2003 &lt;p&gt;&lt;strong&gt;Overview of the work done:&lt;/strong&gt;&lt;/p&gt;
2004 &lt;p&gt;The major time of the phase 1 and 2 were spent in analyzing the input and output paths of the particular protocols being fuzzed. During that time, 5 major protocols of the internet stack were taken up:&lt;/p&gt;
2005 &lt;ol&gt;
2006 &lt;li&gt;IPv4 (Phase 1)&lt;/li&gt;
2007 &lt;li&gt;UDP (Phase 1)&lt;/li&gt;
2008 &lt;li&gt;IPv6 (Phase 2)&lt;/li&gt;
2009 &lt;li&gt;ICMP (Phase 2)&lt;/li&gt;
2010 &lt;li&gt;Ethernet (Phase 2)&lt;/li&gt;
2011 &lt;/ol&gt;
2012 &lt;p&gt;Quite a good amount of time was spent in understanding the input and output processing functions of the particular protocols, the information gathered was to be applied in packet creation code for that protocol. This is important so that we know which parts of the packet can be kept random by the fuzzer based input and which part of the packet need to be set to proper fixed values. Fixing some values in the data packet to be correct is important so that the packet does not get rejected for trivial cases like IP Protocol Version or Internet Checksum. (The procedure to come up with the decisions and the code design and flow is explained in IPv4 Protocol section as an example)&lt;/p&gt;
2013 &lt;p&gt;For each protocol, mainly 2 things needed to be implemented:&amp;nbsp;&lt;/p&gt;
2014 &lt;ol&gt;
2015 &lt;li&gt;The Network Config: the topology for sending and receiving packets example using a TUN device or a TAP device, the socket used and so on. Configuring these devices was the first step in being able to send or receive packets&lt;/li&gt;
2016 &lt;li&gt;Packet Creation: Using the information gathered in the code walkthrough of the protocol functions, packet creation is decided where certain parts of the packet are kept fixed and others random from the fuzzer input itself. Doing so we try to gain maximum code coverage. Also one thing to be noted here, we should not randomly change the fuzzer input, rather do it deterministically following the same rules for each packet, otherwise the evolutionary fuzzer cannot easily grow the corpus.&lt;/li&gt;
2017 &lt;/ol&gt;
2018 &lt;p&gt;In the next section, a few of the protocols will be explained in detail.&lt;/p&gt;
2019 &lt;p&gt;&lt;strong&gt;Protocols&lt;/strong&gt;&lt;/p&gt;
2020 &lt;p&gt;In this section we will talk about the various protocols implemented for fuzzing and talk about the approach taken to create a packet.&amp;nbsp;&lt;/p&gt;
2021 &lt;p&gt;&lt;strong&gt;IPv4:&lt;/strong&gt;&lt;/p&gt;
2022 &lt;p&gt;IPv4 stands for the Internet protocol version 4. It is one of the most widely used protocols in the internet family of protocols. It is used as a network layer protocol for routing and host to host packet delivery using an addressing scheme called IP Address(A 32 bit address scheme). IP Protocol also handles a lot of other functions like fragmentation and reassembly of packets to accommodate for transmission of packets over varying sizes of physical channel capacities. It also supports the concept of multicasting and broadcasting (Via IP Options).&lt;/p&gt;
2023 &lt;p&gt;In order to come up with a strategy for fuzzing, the first step was to carry out a code walkthrough of relevant functions/APIs and data structures involved in the IPv4 protocol. For that the major files and components studied were:&lt;/p&gt;
2024 &lt;ul&gt;
2025 &lt;li&gt;ip_input() =&amp;gt; Which carries out the processing of a incoming packet at the network layer for IPv4 (src &lt;a href=&quot;https://github.com/NetBSD/src/blob/trunk/sys/netinet/ip_input.c&quot;&gt;here&lt;/a&gt;)&lt;/li&gt;
2026 &lt;li&gt;ip_output() =&amp;gt; Which carries out the processing of an outgoing packet at the network layer for IPv4 (src &lt;a href=&quot;https://github.com/NetBSD/src/blob/trunk/sys/netinet/ip_output.c&quot;&gt;here&lt;/a&gt;)&lt;/li&gt;
2027 &lt;li&gt;struct ip =&amp;gt; Represents the IP header (src &lt;a href=&quot;https://github.com/NetBSD/src/blob/trunk/sys/netinet/ip.h&quot;&gt;here&lt;/a&gt;)&lt;/li&gt;
2028 &lt;/ul&gt;
2029 &lt;p&gt;These sections of code represent the working of the input and output processing paths of IPv4 protocol and the struct ip is the main IPv4 header. On top of that other APIs related to mbuf (The NetBSD packet), ip_forward(), IP assembly and fragmentation etc. were also studied in order to determine information about packet structure that could be followed.&lt;/p&gt;
2030 &lt;p&gt;In order to be able to reach these various aspects of the protocol and be able to fuzz it, we went forward with packet creation that took care of basic fields of the IP Header so that it would not get rejected in trivial cases as mentioned before. Hence we went ahead and fixed these fields:&lt;/p&gt;
2031 &lt;ul&gt;
2032 &lt;li&gt;&lt;em&gt;IP Version&lt;/em&gt;: Set it to 0x4 which is a 4 bit value.&lt;/li&gt;
2033 &lt;li&gt;&lt;em&gt;IP Header Len&lt;/em&gt;: Which is set to a value greater than or equal to sizeof(struct ip). Setting this to greater than that allows for IP Options processing.&lt;/li&gt;
2034 &lt;li&gt;&lt;em&gt;IP Len&lt;/em&gt;: Set it to the size of the random buffer passed by fuzzer.&lt;/li&gt;
2035 &lt;li&gt;&lt;em&gt;IP Checksum&lt;/em&gt;: We calculate the correct checksum for the packet using the internet checksum algorithm.&lt;/li&gt;
2036 &lt;/ul&gt;
2037 &lt;p&gt;Other fields were allowed to be populated randomly by fuzzer input. Here is an illustration of the IPv4 header with the fields marked in red as fixed.&lt;/p&gt;
2038 &lt;p&gt;
2039 &lt;img src=&quot;//netbsd.org/~kamil/gsoc_2020/rumpnetfuzz.png&quot;&gt;
2040 &lt;p&gt;The packet creation code lies in the following section inside [&lt;a href=&quot;https://github.com/NJnisarg/fuzznetrump/blob/master/src/helpers/pkt_create.c#L152&quot;&gt;pkt_create.c&lt;/a&gt;]. Another important component is the network configuration [located here &lt;a href=&quot;https://github.com/NJnisarg/fuzznetrump/blob/master/src/helpers/net_config.c#L173&quot;&gt;net_config&lt;/a&gt;] where the code related to configuring a TUN/TAP device is present. All the code uses the rumpkernel exposed APIs and syscalls (prepended with rump_sys_) so as to utilize the rumpkernel while executing the application binary. After packet creation and network config is handled the main fuzzing function is written where a series of steps are followed:&lt;/p&gt;
2041 &lt;ol&gt;
2042 &lt;li&gt;We call rump_init() to initialize the rumpkernel linked via libraries&lt;/li&gt;
2043 &lt;li&gt;We setup the Client and server IP addresses&lt;/li&gt;
2044 &lt;li&gt;We setup the TUN device by calling the network config functions described above&lt;/li&gt;
2045 &lt;li&gt;We create the packet using the packet creation function utilizing the random buffer passed by the fuzzer and transforming that into a semi-random buffer.&lt;/li&gt;
2046 &lt;li&gt;Pass this forged packet into the network stack of the rumpkernel linked with the application binary by calling rump_sys_write on the TUN device setup.&lt;/li&gt;
2047 &lt;/ol&gt;
2048 &lt;p&gt;&lt;strong&gt;IPv6:&lt;/strong&gt;&lt;/p&gt;
2049 &lt;p&gt;IPv6 stands for the Internet protocol version 4. It is the successor of the IPv4 protocol. It came into existence in order to overcome the addressing requirements that could not fit in a 32 bit IPv4 address. It is used as a network layer protocol for routing and host to host packet delivery using an addressing scheme called IPv6 Address(A 128 bit address scheme). It also supports almost similar other functions as IPv4 except some things like fragmentation, broadcast(instead uses multicast).&amp;nbsp;&lt;/p&gt;
2050 &lt;p&gt;In order to be able to reach these various aspects of the protocol and be able to fuzz it, we went forward with packet creation that took care of basic fields of the IP Header so that it would not get rejected in trivial cases as mentioned before. Hence we went ahead and fixed these fields:&lt;/p&gt;
2051 &lt;ul&gt;
2052 &lt;li&gt;&lt;em&gt;IP Version&lt;/em&gt;: Set it to 0x6 which is a 4 bit value.&lt;/li&gt;
2053 &lt;li&gt;&lt;em&gt;IP Hop Limit&lt;/em&gt;: This is an alias for TTL. Set it to a maximum possible value of 255(8 bits).&lt;/li&gt;
2054 &lt;/ul&gt;
2055 &lt;p&gt;Other fields were allowed to be populated randomly by fuzzer input. Allowing the payload len value to be randomly populated allowed processing of various &amp;ldquo;next headers&amp;rdquo; or &amp;rdquo;Extension headers&amp;rdquo;. Extension headers carry optional Internet Layer information, and are placed between the fixed header and the upper-layer protocol header. The headers form a chain, using the Next Header fields. The Next Header field in the fixed header indicates the type of the first extension header; the Next Header field of the last extension header indicates the type of the upper-layer protocol header in the payload of the packet. A further work can be done to set the value of the next header chain and form packets for multiple scenarios with a combination of various next headers.&lt;/p&gt;
2056 &lt;p&gt;&lt;strong&gt;UDP:&lt;/strong&gt;&lt;/p&gt;
2057 &lt;p&gt;UDP stands for User Datagram Protocol. It is one of the simplest protocols and is designed to be simple so that it simply carries payload with minimal overhead. It does not have many options except for checksum information and ports in order to demultiplex the packet to the processes.&amp;nbsp;&lt;/p&gt;
2058 &lt;p&gt;Since UDP runs at the transport layer and hence is wrapped up in an IP header. Since we do not want to fuzz the IP code section, we form a well formed IP header so that the packet does not get rejected in the IP processing section. We only randomize the UDP header using the fuzzer input. We used previously built out IP packet creation utilities to form the IP header and then use the fuzzer input for UDP header.&amp;nbsp;&lt;/p&gt;
2059 &lt;p&gt;In UDP, we fix the following fields:&lt;/p&gt;
2060 &lt;ul&gt;
2061 &lt;li&gt;&lt;em&gt;UDP Checksum&lt;/em&gt;: Set it to zero in order to avoid checksums.&lt;/li&gt;
2062 &lt;/ul&gt;
2063 &lt;p&gt;&lt;strong&gt;ICMP:&lt;/strong&gt;&lt;/p&gt;
2064 &lt;p&gt;ICMP stands for Internet control message protocol. This protocol is sometimes called a sister protocol of IP protocol and is used as a troubleshooting protocol at the network layer. It is used for major 2 purposes:&lt;/p&gt;
2065 &lt;ol&gt;
2066 &lt;li&gt;Error messages&lt;/li&gt;
2067 &lt;li&gt;Request-Reply Queries.&lt;/li&gt;
2068 &lt;/ol&gt;
2069 &lt;p&gt;ICMP has a lot of options and is quite generic in the sense that it handles a lot of error messages and queries. Although ICMP is generally considered at the network layer, it is actually wrapped inside an IP header, hence it has its own protocol number(= 1). Again similar to UDP, we wrap the ICMP headers inside IP headers, hence we do not randomize the IP header and only the ICMP headers using fuzzer input.&lt;/p&gt;
2070 &lt;p&gt;In order to test various ICMP messages and queries, we could not fix values for the &lt;em&gt;type &lt;/em&gt;and &lt;em&gt;code&lt;/em&gt; fields in the ICMP header since they decide the ICMP message type. Also if we allowed random input, most of the packets would get rejected since the number of options of type and code fields are limited and most other values would discard the packet while processing. Hence we came up with a solution where we &lt;em&gt;deterministically&lt;/em&gt; modified the input bits from the fuzzer corresponding to the &lt;em&gt;code&lt;/em&gt; and &lt;em&gt;type&lt;/em&gt; fields. For the &lt;em&gt;type&lt;/em&gt; field we simply took a modulo of the number of types(ICMP_NTYPES macro used here). For the value of &lt;em&gt;code &lt;/em&gt;, we had to fix values in a certain range based on the &lt;em&gt;type &lt;/em&gt;value set already. This technique allowed us to cover all different ICMP message types via the fuzzer input. We also ensured that the input buffer was not modified completely randomly, since that is a bad practice for a feedback-driven fuzzer like ours. Apart from this we fixed the ICMP Checksum field as well by calculating the checksum using the internet checksum algorithm.&lt;/p&gt;
2071 &lt;p&gt;&lt;strong&gt;Ethernet:&lt;/strong&gt;&lt;/p&gt;
2072 &lt;p&gt;Ethernet protocol defined by the IEEE 802.3 standard is a widely used data link layer protocol. The ethernet packet called a frame carries an IP(or the network layer protocol) datagram. The header is simple with Link Layer Addresses called MAC address (used for switching at data link layer which is a part of addressing), for source and destination each of 6 octets(=48 bytes) present, followed by a 4 octet Ethertype and QTag field. This is followed by payload and finally the FCS(frame check sequence) which is a four-octet cyclic redundancy check (CRC) that allows detection of corrupted data within the entire frame as received on the receiver side.&amp;nbsp;&lt;/p&gt;
2073 &lt;p&gt;In case of Ethernet protocol fuzzing, we had to use a TAP device instead of a TUN device, since the TUN device supports passing an IP packet to the network stack, whereas a TAP device accepts an ethernet frame.&amp;nbsp;&lt;/p&gt;
2074 &lt;p&gt;For packet creation, we set the source and destination MAC address and let the payload and ethertype be randomly populated by the fuzzer.&lt;/p&gt;
2075 &lt;p&gt;&lt;br /&gt;&lt;br /&gt;&lt;/p&gt;
2076 &lt;p&gt;&lt;strong&gt;Current Progress and Next steps&lt;/strong&gt;&lt;/p&gt;
2077 &lt;p&gt;The project currently has reached a stage where many major internet family protocols have been covered for fuzzing. As described above a structured approach to fuzzing them have been taken by forming packets based on the internal workings of the protocols. Also as mentioned in the previous post, Rumpkernel environment is being used for fuzzing all these protocols. In order to get better results as compared to raw fuzzing, we have taken these steps. In the next report we shall talk about and compare the coverage of raw fuzzing with our approach.&lt;/p&gt;
2078 &lt;p&gt;For the next phase of GSoC, the major focus would be to validate this process of fuzzing by various methods to check the penetration of packets into the network stack as well as the code coverage. Also the code would be made more streamlined and standardized so that it can be extended for adding more protocols even beyond the scope of the GSoC project.&lt;/p&gt;</content>
2079 </entry>
2080 <entry>
2081 <id>https://blog.netbsd.org/tnf/entry/gsoc_reports_benchmarking_netbsd_second</id>
2082 <title type="html">GSoC Reports: Benchmarking NetBSD, second evaluation report</title>
2083 <author><name>Leonardo Taccari</name></author>
2084 <link rel="alternate" type="text/html" href="https://blog.netbsd.org/tnf/entry/gsoc_reports_benchmarking_netbsd_second"/>
2085 <published>2020-08-12T10:09:16+00:00</published>
2086 <updated>2020-08-12T10:09:16+00:00</updated>
2087 <category term="/General" label="General" />
2088 <category term="gsoc" scheme="http://roller.apache.org/ns/tags/" />
2089 <category term="benchmarking" scheme="http://roller.apache.org/ns/tags/" />
2090 <summary type="html">&lt;p&gt;This report was written by Apurva Nandan as part of Google Summer of Code
2091 2020.&lt;/p&gt;
2092
2093 &lt;p&gt;This blog post is in continuation of
2094 &lt;a href=&quot;//blog.NetBSD.org/tnf/entry/gsoc_reports_benchmarking_netbsd_first&quot;&gt;GSoC Reports: Benchmarking NetBSD, first evaluation report&lt;/a&gt;
2095 blog and describes my progress in the second phase of GSoC 2020 under
2096 The NetBSD Foundation.&lt;/p&gt;
2097
2098 &lt;p&gt;In this phase, I worked on the automation of the regression suite made
2099 using &lt;a href=&quot;https://www.phoronix-test-suite.com&quot;&gt;Phoronix Test Suite (PTS)&lt;/a&gt;
2100 and its integration with &lt;a href=&quot;https://www.gson.org/netbsd/anita&quot;&gt;Anita&lt;/a&gt;.&lt;/p&gt;
2101
2102 &lt;p&gt;The automation framework consists of two components Phoromatic server,
2103 provided by Phoronix Test Suite in pkgsrc, and Anita, a Python tool for
2104 automating NetBSD installation.&lt;/p&gt;
2105 </summary>
2106 <content type="html">&lt;p&gt;This report was written by Apurva Nandan as part of Google Summer of Code
2107 2020.&lt;/p&gt;
2108
2109 &lt;p&gt;This blog post is in continuation of
2110 &lt;a href=&quot;//blog.NetBSD.org/tnf/entry/gsoc_reports_benchmarking_netbsd_first&quot;&gt;GSoC Reports: Benchmarking NetBSD, first evaluation report&lt;/a&gt;
2111 blog and describes my progress in the second phase of GSoC 2020 under
2112 The NetBSD Foundation.&lt;/p&gt;
2113
2114 &lt;p&gt;In this phase, I worked on the automation of the regression suite made
2115 using &lt;a href=&quot;https://www.phoronix-test-suite.com&quot;&gt;Phoronix Test Suite (PTS)&lt;/a&gt;
2116 and its integration with &lt;a href=&quot;https://www.gson.org/netbsd/anita&quot;&gt;Anita&lt;/a&gt;.&lt;/p&gt;
2117
2118 &lt;p&gt;The automation framework consists of two components Phoromatic server,
2119 provided by Phoronix Test Suite in pkgsrc, and Anita, a Python tool for
2120 automating NetBSD installation.&lt;/p&gt;
2121
2122 &lt;h2&gt;About Phoromatic&lt;/h2&gt;
2123
2124 &lt;p&gt;Phoromatic is a remote management system for the Phoronix Test Suite,
2125 which allows the automatic scheduling of tests, remote installation of
2126 new tests, and the management of multiple test systems through a web
2127 interface. Tests can be scheduled to run on a routine basis across
2128 multiple test systems automatically. Phoromatic can also interface with
2129 revision control systems to offer support for issuing new tests on a
2130 context-basis, such as whenever a Git commit has been pushed. The test
2131 results are then available from the web interface.&lt;/p&gt;
2132
2133 &lt;h3&gt;Phoromatic client-server architecture&lt;/h3&gt;
2134
2135 &lt;p&gt;&lt;img src=&quot;//www.NetBSD.org/~leot/blog-posts/imgs/pts_phoromatic-server-client-arch.png&quot; alt=&quot;Phoromatic server-client architecture&quot; width=&quot;800&quot; /&gt;&lt;/p&gt;
2136
2137 &lt;p&gt;The Phoromatic server relies upon a PHP/HHVM built-in web server
2138 process and a PTS-hosted WebSocket server. The web server process
2139 handles the web UI and the responsibilities of the Phoromatic server.&lt;/p&gt;
2140
2141 &lt;p&gt;Phoromatic clients are testing machines installed with PTS that connect
2142 to the Phoromatic web server through the HTTP port of the server.&lt;/p&gt;
2143
2144 &lt;h3&gt;Phoromatic Setup&lt;/h3&gt;
2145
2146 &lt;p&gt;To start the Phoromatic server, Phoromatic server HTTP port and web
2147 server socket port needs to be set in
2148 &lt;code&gt;~/.phoronix-test-suite/user-config.xml&lt;/code&gt; as shown:&lt;/p&gt;
2149
2150 &lt;pre&gt;
2151 ...
2152 &amp;lt;Server&amp;gt;
2153 &amp;lt;RemoteAccessPort&amp;gt;8640&amp;lt;/RemoteAccessPort&amp;gt;
2154 &amp;lt;Password&amp;gt;&amp;lt;/Password&amp;gt;
2155 &amp;lt;WebSocketPort&amp;gt;8642&amp;lt;/WebSocketPort&amp;gt;
2156 &amp;lt;AdvertiseServiceZeroConf&amp;gt;TRUE&amp;lt;/AdvertiseServiceZeroConf&amp;gt;
2157 &amp;lt;AdvertiseServiceOpenBenchmarkRelay&amp;gt;TRUE&amp;lt;/AdvertiseServiceOpenBenchmarkRelay&amp;gt;
2158 &amp;lt;PhoromaticStorage&amp;gt;~/.phoronix-test-suite/phoromatic/&amp;lt;/PhoromaticStorage&amp;gt;
2159 &amp;lt;/Server&amp;gt;
2160 &lt;/pre&gt;
2161
2162 &lt;h3&gt;Phoromatic Usage&lt;/h3&gt;
2163
2164 &lt;p&gt;To start the Phoromatic web server for controlling local Phoronix Test Suite client systems:&lt;/p&gt;
2165
2166 &lt;p&gt;&lt;code&gt;$ phoronix-test-suite start-phoromatic-server&lt;/code&gt;&lt;/p&gt;
2167
2168 &lt;p&gt;The Phoromatic web server will be hosted at &lt;code&gt;localhost:8640&lt;/code&gt; and will require a local account creation on the server.&lt;/p&gt;
2169
2170 &lt;h4&gt;Phoromatic Clients&lt;/h4&gt;
2171
2172 &lt;p&gt;The Phoromatic client is used for connecting to a Phoromatic server to facilitate the automatic running of tests on that client.&lt;/p&gt;
2173
2174 &lt;p&gt;Phoromatic clients can be created and connected to the server using the following command:&lt;/p&gt;
2175
2176 &lt;p&gt;&lt;code&gt;
2177 $ phoronix-test-suite phoromatic.connect SERVER_IP:SERVER_HTTP_PORT/ACCOUNT_ID
2178 &lt;/code&gt;&lt;/p&gt;
2179
2180 &lt;p&gt;Phoromatic server interacts with the Phoromatic clients through the HTTP port specified in the &lt;code&gt;~/.phoronix-test-suite/user-config.xml&lt;/code&gt;.&lt;/p&gt;
2181
2182 &lt;h4&gt;Phoromatic Test-schedules&lt;/h4&gt;
2183
2184 &lt;p&gt;A test schedule is used to facilitate automatically running a set of
2185 test(s)/suite(s) on either a routine timed basis or whenever triggered
2186 by an external script or process, e.g. Git/VCS commit, manually
2187 triggered, etc.
2188 Phoromatic provides an option for pre-install, pre-run, post-install
2189 and post-run shell scripts that are executed on the Phoromatic
2190 clients.
2191 Test-schedules can be configured to run any tests on any specific
2192 systems.&lt;/p&gt;
2193
2194 &lt;h2&gt;About Anita&lt;/h2&gt;
2195
2196 &lt;p&gt;Anita is a tool for automated testing of the NetBSD operating system.
2197 Using Anita, we can download a NetBSD distribution and install it in a
2198 virtual machine in a fully automated fashion. Anita is written in
2199 Python and uses the pexpect module to &amp;ldquo;screen scrape&amp;rdquo; the sysinst
2200 output over an emulated serial console and script the installation
2201 procedure.&lt;/p&gt;
2202
2203 &lt;h3&gt;Installation&lt;/h3&gt;
2204
2205 &lt;p&gt;Anita can be installed on NetBSD, Linux and macOS systems using the following:&lt;/p&gt;
2206
2207 &lt;pre&gt;
2208 $ pip install pexpect
2209 $ git clone https://github.com/gson1703/anita/
2210 $ python setup.py install
2211 &lt;/pre&gt;
2212
2213 &lt;h2&gt;Phoromatic-Anita Integration&lt;/h2&gt;
2214
2215 &lt;p&gt;I would like to describe the workflow here briefly:&lt;/p&gt;
2216
2217 &lt;ul&gt;
2218 &lt;li&gt;A test-schedule was created on the Phoromatic server meant to run
2219 &lt;code&gt;pts/idle-1.2.0&lt;/code&gt; test on the host machine that contains the
2220 &lt;a href=&quot;https://gist.github.com/apurvanandan1997/48b54402db1df3723920735f85fc7934&quot;&gt;phoromatic-anita-integration.sh&lt;/a&gt;
2221 as a pre-run script.&lt;/li&gt;
2222 &lt;li&gt;The script performs the following:
2223
2224 &lt;ul&gt;
2225 &lt;li&gt;Creates a mountable disk image with an executable script for
2226 setting up Phoronix Test Suite and Phoromatic client creation on the
2227 benchmarking VM systems.&lt;/li&gt;
2228 &lt;li&gt;Invokes Anita with the appropriate command-line options for
2229 configurations and network setup and mounts the image to run the
2230 configuration script on the VM.&lt;/li&gt;
2231 &lt;li&gt;Configuration script performs hostname change, DHCP setup, NFS
2232 setup, &lt;code&gt;PKG_PATH&lt;/code&gt; setup, PTS installation, its configuration and
2233 connecting it to the Phoromatic server through a network bridge.&lt;/li&gt;
2234 &lt;/ul&gt;
2235 &lt;/li&gt;
2236 &lt;li&gt;Once the benchmarking VM systems get connected to the Phoromatic
2237 server, Phoromatic server identifies the benchmarking VM systems with
2238 their IP address, hostname and MAC address.&lt;/li&gt;
2239 &lt;li&gt;After the identification, Phoromatic initiates the pending tests on
2240 VM (test-profiles are downloaded on the go in the VM and executed) and
2241 maintains a history of the test result data.&lt;/li&gt;
2242 &lt;/ul&gt;
2243
2244
2245 &lt;p&gt;Few points to be noted:&lt;/p&gt;
2246
2247 &lt;ul&gt;
2248 &lt;li&gt;I have used a local PKG_PATH with a NFS server setup as PTS 9.6.1 is
2249 available in wip and recompiling it would be a wastage of time. Later I
2250 have planned to use the binary shard by Joyent:
2251 &lt;a href=&quot;https://pkgsrc.joyent.com/packages/NetBSD/9.99.69/amd64/All/&quot;&gt;https://pkgsrc.joyent.com/packages/NetBSD/9.99.69/amd64/All/&lt;/a&gt; once the
2252 updated PTS gets upstreamed.&lt;/li&gt;
2253 &lt;li&gt;The host machine needs some one-time manual setup like installation
2254 of QEMU, Anita, pexpect, term, cvs, etc., initial user registration on
2255 Phoromatic server, Phoromatic port setup, network bridge setup. Apart
2256 from this, the rest of the framework does not require user
2257 supervision.&lt;/li&gt;
2258 &lt;/ul&gt;
2259
2260
2261 &lt;h5&gt;VM configuration script&lt;/h5&gt;
2262
2263 &lt;p&gt;The following script is used as a pre-run script in the test-schedules for invoking Anita and setting up the VMs:&lt;/p&gt;
2264
2265 &lt;p&gt;&lt;a href=&quot;https://gist.github.com/apurvanandan1997/48b54402db1df3723920735f85fc7934&quot;&gt;https://gist.github.com/apurvanandan1997/48b54402db1df3723920735f85fc7934&lt;/a&gt;&lt;/p&gt;
2266
2267 &lt;h5&gt;Networking Setup&lt;/h5&gt;
2268
2269 &lt;p&gt;A bridged networking mode configuration of QEMU has been used in Anita
2270 as multiple VMs will be able to accommodate with a single bridge
2271 (created on the host machine, one-time setup) using
2272 &lt;a href=&quot;//man.NetBSD.org/dhcpcd.8&quot;&gt;dhcpcd(8)&lt;/a&gt;,
2273 without complicated host forwarding setup (Phoromatic server requires
2274 HTTP port forwarding).&lt;/p&gt;
2275
2276 &lt;p&gt;In order to enable bridged networking for your QEMU guests, you must
2277 first create and configure a bridge interface on your host.&lt;/p&gt;
2278
2279 &lt;pre&gt;
2280 # ifconfig virbr0 create
2281 &lt;/pre&gt;
2282
2283 &lt;p&gt;Next, you must specify the newly-created bridge interface in
2284 &lt;code&gt;/etc/qemu/bridge.conf&lt;/code&gt;:&lt;/p&gt;
2285
2286 &lt;pre&gt;
2287 $ sudo mkdir /etc/qemu
2288 $ sudo touch /etc/qemu/bridge.conf &amp;amp;&amp;amp; sudo chmod 644 /etc/qemu/bridge.conf
2289 $ sudo sh -c &quot;echo &apos;allow virbr0&apos; &gt;&gt; /etc/qemu/bridge.conf&quot;
2290 &lt;/pre&gt;
2291
2292 &lt;p&gt;Finally, in order for non-privileged processes to be able to invoke
2293 qemu-bridge-helper, you must set the setuid bit on the utility:&lt;/p&gt;
2294
2295 &lt;p&gt;&lt;code&gt;
2296 $ sudo chmod u+s /usr/local/libexec/qemu-bridge-helper
2297 &lt;/code&gt;&lt;/p&gt;
2298
2299 &lt;p&gt;For more details on the bridged mode networking setup in QEMU, please
2300 refer to the following guides:&lt;/p&gt;
2301
2302 &lt;ul&gt;
2303 &lt;li&gt;&lt;a href=&quot;https://t.pagef.lt/basic-networking-with-qemu/&quot;&gt;https://t.pagef.lt/basic-networking-with-qemu/&lt;/a&gt;&lt;/li&gt;
2304 &lt;li&gt;&lt;a href=&quot;https://www.NetBSD.org/docs/guide/en/chap-net-practice.html#chap-net-practice-bridge&quot;&gt;https://www.NetBSD.org/docs/guide/en/chap-net-practice.html#chap-net-practice-bridge&lt;/a&gt;&lt;/li&gt;
2305 &lt;/ul&gt;
2306
2307
2308 &lt;h5&gt;Reproducing the framework&lt;/h5&gt;
2309
2310 &lt;p&gt;To reproduce the framework, you need to have Phoronix Test Suite, QEMU,
2311 Anita, pexpect, cvs, xterm, makefs installed on your host machine.&lt;/p&gt;
2312
2313 &lt;p&gt;For example on NetBSD:&lt;/p&gt;
2314
2315 &lt;pre&gt;
2316 # pkg_add qemu
2317 # pkg_add py37-anita
2318 $ cd pkgsrc/wip/phoronix-test-suite
2319 $ make install
2320 &lt;/pre&gt;
2321
2322 &lt;p&gt;The step-by-step process to get the framework after installing PTS,
2323 including the one-time manual setup, can be summarized as follows: All
2324 control and configuration of the Phoromatic Server is done via the
2325 web-based interface when the Phoromatic Server is active.&lt;/p&gt;
2326
2327 &lt;ul&gt;
2328 &lt;li&gt;Configure the port of Phoromatic server as 8640 and web socket as
2329 8642 as described above.&lt;/li&gt;
2330 &lt;li&gt;Start the Phoromatic server using the command stated above.
2331 &lt;img src=&quot;//www.NetBSD.org/~leot/blog-posts/imgs/pts_phoromatic-register.png&quot; alt=&quot;Phoromatic login page image&quot; width=&quot;800&quot; /&gt;&lt;/li&gt;
2332 &lt;li&gt;Create your user account on the Phoromatic server using the web interface GUI.
2333 &lt;img src=&quot;//www.NetBSD.org/~leot/blog-posts/imgs/pts_phoromatic.png&quot; alt=&quot;Phoromatic default page image&quot; width=&quot;800&quot; /&gt;&lt;/li&gt;
2334 &lt;li&gt;Disable client system approval for new system addition from the
2335 settings menu in the web interface.&lt;/li&gt;
2336 &lt;li&gt;Connect the host machine as a Phoromatic client to the Phoromatic
2337 server using the command stated above.&lt;/li&gt;
2338 &lt;li&gt;Create a test-schedule for the host machine with the
2339 &lt;a href=&quot;https://gist.github.com/apurvanandan1997/48b54402db1df3723920735f85fc7934&quot;&gt;pre-run script&lt;/a&gt;
2340 as specified above and &lt;code&gt;pts/idle-1.2.0&lt;/code&gt; as the test-profile.
2341 &lt;img src=&quot;//www.NetBSD.org/~leot/blog-posts/imgs/pts_create-sch.png&quot; alt=&quot;Phoromatic create test-schedule image&quot; width=&quot;800&quot; /&gt;&lt;/li&gt;
2342 &lt;li&gt;Execute the test-schedule or assign it on a timed-schedule and watch it running!
2343 &lt;img src=&quot;//www.NetBSD.org/~leot/blog-posts/imgs/pts_host-machine-test-sch.png&quot; alt=&quot;Phoromatic Anita integration test-scehdule&quot; width=&quot;800&quot; /&gt;&lt;/li&gt;
2344 &lt;li&gt;New VM systems with the latest NetBSD-current binaries and packages
2345 will be created and identified by Phoromatic server automatically.&lt;/li&gt;
2346 &lt;li&gt;Once for all, we need to specify what benchmarking test-profiles need
2347 to be run on the VM systems in the test-schedules section and it will
2348 be taken care of by Phoromatic.
2349 &lt;img src=&quot;//www.NetBSD.org/~leot/blog-posts/imgs/pts_test-sch.png&quot; alt=&quot;Phoromatic benchmarking system test-schedule image&quot; width=&quot;800&quot; /&gt;&lt;/li&gt;
2350 &lt;li&gt;The result history can also be viewed from Phoromatic web interface.&lt;/li&gt;
2351 &lt;/ul&gt;
2352
2353
2354 &lt;p&gt;You can have a look at the video to get a clearer picture of how to
2355 setup the framework:&lt;/p&gt;
2356
2357 &lt;iframe width=&quot;800&quot; height=&quot;450&quot; src=&quot;https://www.youtube.com/embed/7OSaqDWIsxo&quot; frameborder=&quot;0&quot;&gt;&lt;/iframe&gt;
2358
2359 &lt;h2&gt;Future Plans&lt;/h2&gt;
2360
2361 &lt;p&gt;The regression suite is complete and final tasks of deploying it on
2362 benchmark.NetBSD.org and upstreaming the wip of Phoronix Test Suite
2363 will be done in the final phase of my GSoC project.
2364 I want to thank my mentors for their constant support.&lt;/p&gt;
2365 </content>
2366 </entry>
2367 <entry>
2368 <id>https://blog.netbsd.org/tnf/entry/gsoc_2020_second_evaluation_report</id>
2369 <title type="html">GSoC 2020 Second Evaluation Report: Curses Library Automated Testing</title>
2370 <author><name>Kamil Rytarowski</name></author>
2371 <link rel="alternate" type="text/html" href="https://blog.netbsd.org/tnf/entry/gsoc_2020_second_evaluation_report"/>
2372 <published>2020-08-07T11:21:00+00:00</published>
2373 <updated>2020-08-07T11:47:52+00:00</updated>
2374 <category term="/The NetBSD Foundation" label="The NetBSD Foundation" />
2375 <category term="curses" scheme="http://roller.apache.org/ns/tags/" />
2376 <summary type="html">&lt;i&gt;This report was prepared by Naman Jain as a part of Google Summer of Code 2020&lt;/i&gt;
2377 &lt;p&gt;My GSoC project under NetBSD involves the development of test framework of curses library. This blog report is second in series of blog reports; you can have a look at the &lt;a href=&quot;https://blog.netbsd.org/tnf/entry/gsoc_reports_curses_library_automated&quot; rel=&quot;nofollow&quot;&gt;first report&lt;/a&gt;. This report would cover the progress made in second coding phase along with providing some insights into the libcurses.&lt;/p&gt;</summary>
2378 <content type="html">&lt;i&gt;This report was prepared by Naman Jain as a part of Google Summer of Code 2020&lt;/i&gt;
2379 &lt;p&gt;My GSoC project under NetBSD involves the development of test framework of curses library. This blog report is second in series of blog reports; you can have a look at the &lt;a href=&quot;https://blog.netbsd.org/tnf/entry/gsoc_reports_curses_library_automated&quot; rel=&quot;nofollow&quot;&gt;first report&lt;/a&gt;. This report would cover the progress made in second coding phase along with providing some insights into the libcurses.&lt;/p&gt;
2380 &lt;h2&gt;&lt;a id=&quot;user-content-complex-characters&quot; class=&quot;anchor&quot; aria-hidden=&quot;true&quot; href=&quot;#complex-characters&quot;&gt;&lt;svg class=&quot;octicon octicon-link&quot; viewBox=&quot;0 0 16 16&quot; version=&quot;1.1&quot; width=&quot;16&quot; height=&quot;16&quot; aria-hidden=&quot;true&quot;&gt;&lt;path fill-rule=&quot;evenodd&quot; d=&quot;M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z&quot;&gt;&lt;/path&gt;&lt;/svg&gt;&lt;/a&gt;Complex characters&lt;/h2&gt;
2381 &lt;p&gt;A complex character is a set of associated character, which may include a spacing character and non-spacing characters associated with it. Typical effects of non-spacing character on associated complex character &lt;em&gt;c&lt;/em&gt; include: modifying the appearance of &lt;em&gt;c&lt;/em&gt; (like adding diacritical marks) or bridge &lt;em&gt;c&lt;/em&gt; with the following character.
2382 The &lt;strong&gt;cchar_t&lt;/strong&gt; data type represents a complex character and its rendition. In NetBSD, this data type has following structure:&lt;/p&gt;
2383 &lt;div class=&quot;highlight highlight-source-c&quot;&gt;&lt;pre&gt;&lt;span class=&quot;pl-k&quot;&gt;struct&lt;/span&gt; &lt;span class=&quot;pl-c1&quot;&gt;cchar_t&lt;/span&gt; {
2384 &lt;span class=&quot;pl-c1&quot;&gt;attr_t&lt;/span&gt; attributes; &lt;span class=&quot;pl-c&quot;&gt;&lt;span class=&quot;pl-c&quot;&gt;/*&lt;/span&gt; character attributes &lt;span class=&quot;pl-c&quot;&gt;*/&lt;/span&gt;&lt;/span&gt;
2385 &lt;span class=&quot;pl-k&quot;&gt;unsigned&lt;/span&gt; elements; &lt;span class=&quot;pl-c&quot;&gt;&lt;span class=&quot;pl-c&quot;&gt;/*&lt;/span&gt; number of wide char in vals&lt;span class=&quot;pl-c&quot;&gt;*/&lt;/span&gt;&lt;/span&gt;
2386 &lt;span class=&quot;pl-c1&quot;&gt;wchar_t&lt;/span&gt; vals[CURSES_CCHAR_MAX]; &lt;span class=&quot;pl-c&quot;&gt;&lt;span class=&quot;pl-c&quot;&gt;/*&lt;/span&gt; wide chars including non-spacing &lt;span class=&quot;pl-c&quot;&gt;*/&lt;/span&gt;&lt;/span&gt;
2387 };&lt;/pre&gt;&lt;/div&gt;
2388 &lt;p&gt;&lt;em&gt;vals&lt;/em&gt; array contains the spacing character and associated non-spacing characters. Note that NetBSD supports &lt;strong&gt;wchar_t&lt;/strong&gt; (wide character) due to which multi-byte characters are supported. To use the complex characters one has to correctly set the locale settings.
2389 In this coding period, I wrote tests for routines involving complex characters.&lt;/p&gt;
2390 &lt;h2&gt;&lt;a id=&quot;user-content-alternate-character-set&quot; class=&quot;anchor&quot; aria-hidden=&quot;true&quot; href=&quot;#alternate-character-set&quot;&gt;&lt;svg class=&quot;octicon octicon-link&quot; viewBox=&quot;0 0 16 16&quot; version=&quot;1.1&quot; width=&quot;16&quot; height=&quot;16&quot; aria-hidden=&quot;true&quot;&gt;&lt;path fill-rule=&quot;evenodd&quot; d=&quot;M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z&quot;&gt;&lt;/path&gt;&lt;/svg&gt;&lt;/a&gt;Alternate character set&lt;/h2&gt;
2391 &lt;p&gt;When you print &quot;BSD&quot;, you would send the hex-codes 42, 53, 44 to the terminal. Capability of graphic capable printers was limited by 8-bit ASCII code. To solve this, additional character sets were introduced. We can switch between the modes using escape sequence. One such character set for Special Graphics is used by curses for line drawing. In a shell you can type&lt;/p&gt;
2392 &lt;div class=&quot;highlight highlight-source-shell&quot;&gt;&lt;pre&gt;&lt;span class=&quot;pl-c1&quot;&gt;echo&lt;/span&gt; -e &lt;span class=&quot;pl-s&quot;&gt;&lt;span class=&quot;pl-pds&quot;&gt;&quot;&lt;/span&gt;\e(0j\e(b&lt;span class=&quot;pl-pds&quot;&gt;&quot;&lt;/span&gt;&lt;/span&gt;&lt;/pre&gt;&lt;/div&gt;
2393 &lt;p&gt;to get a lower-right corner glyph. This enables alternate character mode (&lt;code&gt;\e(&lt;/code&gt;), prints a character(&lt;code&gt;j&lt;/code&gt;) and disables alternate character mode (&lt;code&gt;\e(b&lt;/code&gt;). One might wonder where this &apos;j&apos; to &apos;Lower Right Corner glyph&apos; comes from. You may see that mapping (&quot;acsc=``aaffggiijjkkllmmnnooppqqrrssttuuvvwwxxyyzz{{||}}~~,) via&lt;/p&gt;
2394 &lt;div class=&quot;highlight highlight-source-shell&quot;&gt;&lt;pre&gt;infocmp -1 &lt;span class=&quot;pl-smi&quot;&gt;$TERM&lt;/span&gt; &lt;span class=&quot;pl-k&quot;&gt;|&lt;/span&gt; grep acsc&lt;/pre&gt;&lt;/div&gt;
2395 &lt;p&gt;These characters are used in &lt;code&gt;box_set()&lt;/code&gt;, &lt;code&gt;border_set()&lt;/code&gt;, etc. functions which I tested in the second coding period.&lt;/p&gt;
2396 &lt;h2&gt;&lt;a id=&quot;user-content-progress-in-the-second-coding-phase&quot; class=&quot;anchor&quot; aria-hidden=&quot;true&quot; href=&quot;#progress-in-the-second-coding-phase&quot;&gt;&lt;svg class=&quot;octicon octicon-link&quot; viewBox=&quot;0 0 16 16&quot; version=&quot;1.1&quot; width=&quot;16&quot; height=&quot;16&quot; aria-hidden=&quot;true&quot;&gt;&lt;path fill-rule=&quot;evenodd&quot; d=&quot;M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z&quot;&gt;&lt;/path&gt;&lt;/svg&gt;&lt;/a&gt;Progress in the second coding phase&lt;/h2&gt;
2397 &lt;h3&gt;&lt;a id=&quot;user-content-improvements-in-the-framework&quot; class=&quot;anchor&quot; aria-hidden=&quot;true&quot; href=&quot;#improvements-in-the-framework&quot;&gt;&lt;svg class=&quot;octicon octicon-link&quot; viewBox=&quot;0 0 16 16&quot; version=&quot;1.1&quot; width=&quot;16&quot; height=&quot;16&quot; aria-hidden=&quot;true&quot;&gt;&lt;path fill-rule=&quot;evenodd&quot; d=&quot;M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z&quot;&gt;&lt;/path&gt;&lt;/svg&gt;&lt;/a&gt;Improvements in the framework:&lt;/h3&gt;
2398 &lt;ol&gt;
2399 &lt;li&gt;Added support for testing of functions to be called before &lt;code&gt;initscr()&lt;/code&gt;&lt;/li&gt;
2400 &lt;li&gt;Updated the unsupported function definitions with some minor bug fixes.&lt;/li&gt;
2401 &lt;/ol&gt;
2402 &lt;h3&gt;&lt;a id=&quot;user-content-testing-and-bug-reports&quot; class=&quot;anchor&quot; aria-hidden=&quot;true&quot; href=&quot;#testing-and-bug-reports&quot;&gt;&lt;svg class=&quot;octicon octicon-link&quot; viewBox=&quot;0 0 16 16&quot; version=&quot;1.1&quot; width=&quot;16&quot; height=&quot;16&quot; aria-hidden=&quot;true&quot;&gt;&lt;path fill-rule=&quot;evenodd&quot; d=&quot;M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z&quot;&gt;&lt;/path&gt;&lt;/svg&gt;&lt;/a&gt;Testing and bug reports&lt;/h3&gt;
2403 &lt;ol&gt;
2404 &lt;li&gt;Added tests for following families of functions:
2405 &lt;ul&gt;
2406 &lt;li&gt;Complex character routines.&lt;/li&gt;
2407 &lt;li&gt;Line/box drawing routines.&lt;/li&gt;
2408 &lt;li&gt;Pad routines.&lt;/li&gt;
2409 &lt;li&gt;Window and sub-window operations.&lt;/li&gt;
2410 &lt;li&gt;Curson manipulation routines&lt;/li&gt;
2411 &lt;/ul&gt;
2412 &lt;/li&gt;
2413 &lt;li&gt;Reported bugs (and possible fixes if I know):
2414 &lt;ul&gt;
2415 &lt;li&gt;&lt;a href=&quot;https://gnats.netbsd.org/cgi-bin/query-pr-single.pl?number=55454&quot; rel=&quot;nofollow&quot;&gt;lib/55454&lt;/a&gt; &lt;code&gt;wredrawln()&lt;/code&gt; in libcurses does not follow the sensible behaviour [&lt;em&gt;fixed&lt;/em&gt;]&lt;/li&gt;
2416 &lt;li&gt;&lt;a href=&quot;https://gnats.netbsd.org/cgi-bin/query-pr-single.pl?number=55460&quot; rel=&quot;nofollow&quot;&gt;lib/55460&lt;/a&gt; copy error in libcurses [&lt;em&gt;fixed&lt;/em&gt;]&lt;/li&gt;
2417 &lt;li&gt;&lt;a href=&quot;https://gnats.netbsd.org/cgi-bin/query-pr-single.pl?number=55474&quot; rel=&quot;nofollow&quot;&gt;lib/55474&lt;/a&gt; &lt;code&gt;wattroff()&lt;/code&gt; unsets all attributes if passed STANDOUT as argument [&lt;em&gt;standard is not clear, so decided to have as it is&lt;/em&gt;]&lt;/li&gt;
2418 &lt;li&gt;&lt;a href=&quot;https://gnats.netbsd.org/cgi-bin/query-pr-single.pl?number=55482&quot; rel=&quot;nofollow&quot;&gt;lib/55482&lt;/a&gt; &lt;code&gt;slk_restore()&lt;/code&gt; does not restore the slk screen&lt;/li&gt;
2419 &lt;li&gt;&lt;a href=&quot;https://gnats.netbsd.org/cgi-bin/query-pr-single.pl?number=55484&quot; rel=&quot;nofollow&quot;&gt;lib/55484&lt;/a&gt; &lt;code&gt;newwin()&lt;/code&gt; results into seg fault [&lt;em&gt;fixed&lt;/em&gt;]&lt;/li&gt;
2420 &lt;li&gt;&lt;a href=&quot;https://gnats.netbsd.org/cgi-bin/query-pr-single.pl?number=55496&quot; rel=&quot;nofollow&quot;&gt;lib/55496&lt;/a&gt; &lt;code&gt;bkgrnd()&lt;/code&gt; doesn&apos;t work as expected&lt;/li&gt;
2421 &lt;li&gt;&lt;a href=&quot;https://gnats.netbsd.org/cgi-bin/query-pr-single.pl?number=55517&quot; rel=&quot;nofollow&quot;&gt;lib/55517&lt;/a&gt; &lt;code&gt;wresize()&lt;/code&gt; function incorrectly resizes the subwindows&lt;/li&gt;
2422 &lt;/ul&gt;
2423 &lt;/li&gt;
2424 &lt;/ol&gt;
2425 &lt;p&gt;I would like to thank my mentors Brett and Martin, as well as the NetBSD community for their support whenever I faced some issues.&lt;/p&gt;</content>
2426 </entry>
2427 <entry>
2428 <id>https://blog.netbsd.org/tnf/entry/gsoc_reports_fuzzing_rumpkernel_syscalls1</id>
2429 <title type="html">GSoC Reports: Fuzzing Rumpkernel Syscalls, Part 2</title>
2430 <author><name>Kamil Rytarowski</name></author>
2431 <link rel="alternate" type="text/html" href="https://blog.netbsd.org/tnf/entry/gsoc_reports_fuzzing_rumpkernel_syscalls1"/>
2432 <published>2020-08-05T08:42:37+00:00</published>
2433 <updated>2020-08-05T08:42:37+00:00</updated>
2434 <category term="/The NetBSD Foundation" label="The NetBSD Foundation" />
2435 <category term="fuzzing" scheme="http://roller.apache.org/ns/tags/" />
2436 <category term="rump" scheme="http://roller.apache.org/ns/tags/" />
2437 <category term="gsoc" scheme="http://roller.apache.org/ns/tags/" />
2438 <summary type="html">&lt;i&gt;This report was prepared by Aditya Vardhan Padala as a part of Google Summer of Code 2020&lt;/i&gt;
2439 &lt;p&gt;I have been working on Fuzzing Rumpkernel Syscalls. This blogpost details the work I have done during my second coding period.&lt;/p&gt;</summary>
2440 <content type="html">&lt;i&gt;This report was prepared by Aditya Vardhan Padala as a part of Google Summer of Code 2020&lt;/i&gt;
2441 &lt;p&gt;I have been working on Fuzzing Rumpkernel Syscalls. This blogpost details the work I have done during my second coding period.&lt;/p&gt;
2442 &lt;h2 id=&quot;reproducing-crash-found-in-ioctl-&quot;&gt;Reproducing crash found in ioctl()&lt;/h2&gt;
2443 &lt;p&gt;Kamil has worked on reproducing the following crash &lt;/p&gt;
2444 &lt;pre&gt;&lt;code class=&quot;lang-bash=&quot;&gt;Thread &lt;span class=&quot;hljs-number&quot;&gt;1&lt;/span&gt; &lt;span class=&quot;hljs-string&quot;&gt;&quot;&quot;&lt;/span&gt; received signal SIGSEGV, Segmentation fault.
2445 pipe_ioctl (fp=&amp;lt;optimized &lt;span class=&quot;hljs-keyword&quot;&gt;out&lt;/span&gt;&amp;gt;, cmd=&amp;lt;optimized &lt;span class=&quot;hljs-keyword&quot;&gt;out&lt;/span&gt;&amp;gt;, data=&lt;span class=&quot;hljs-number&quot;&gt;0x7f7fffccd700&lt;/span&gt;)
2446 at /usr/src/&lt;span class=&quot;hljs-class&quot;&gt;&lt;span class=&quot;hljs-keyword&quot;&gt;lib&lt;/span&gt;/&lt;span class=&quot;hljs-title&quot;&gt;librump&lt;/span&gt;/../../&lt;span class=&quot;hljs-title&quot;&gt;sys&lt;/span&gt;/&lt;span class=&quot;hljs-title&quot;&gt;rump&lt;/span&gt;/../&lt;span class=&quot;hljs-title&quot;&gt;kern&lt;/span&gt;/&lt;span class=&quot;hljs-title&quot;&gt;sys_pipe&lt;/span&gt;.&lt;span class=&quot;hljs-title&quot;&gt;c&lt;/span&gt;:1108&lt;/span&gt;
2447 &lt;span class=&quot;hljs-symbol&quot;&gt;warning:&lt;/span&gt; Source file is more recent than executable.
2448 &lt;span class=&quot;hljs-number&quot;&gt;1108&lt;/span&gt; *(int *)data = pipe-&amp;gt;pipe_buffer.cnt;
2449 (gdb) bt
2450 &lt;span class=&quot;hljs-comment&quot;&gt;#0 pipe_ioctl (fp=&amp;lt;optimized out&amp;gt;, cmd=&amp;lt;optimized out&amp;gt;, data=0x7f7fffccd700)&lt;/span&gt;
2451 at /usr/src/&lt;span class=&quot;hljs-class&quot;&gt;&lt;span class=&quot;hljs-keyword&quot;&gt;lib&lt;/span&gt;/&lt;span class=&quot;hljs-title&quot;&gt;librump&lt;/span&gt;/../../&lt;span class=&quot;hljs-title&quot;&gt;sys&lt;/span&gt;/&lt;span class=&quot;hljs-title&quot;&gt;rump&lt;/span&gt;/../&lt;span class=&quot;hljs-title&quot;&gt;kern&lt;/span&gt;/&lt;span class=&quot;hljs-title&quot;&gt;sys_pipe&lt;/span&gt;.&lt;span class=&quot;hljs-title&quot;&gt;c&lt;/span&gt;:1108&lt;/span&gt;
2452 &lt;span class=&quot;hljs-comment&quot;&gt;#1 0x000075b0de65083f in sys_ioctl (l=&amp;lt;optimized out&amp;gt;, uap=0x7f7fffccd820, retval=&amp;lt;optimized out&amp;gt;)&lt;/span&gt;
2453 at /usr/src/&lt;span class=&quot;hljs-class&quot;&gt;&lt;span class=&quot;hljs-keyword&quot;&gt;lib&lt;/span&gt;/&lt;span class=&quot;hljs-title&quot;&gt;librump&lt;/span&gt;/../../&lt;span class=&quot;hljs-title&quot;&gt;sys&lt;/span&gt;/&lt;span class=&quot;hljs-title&quot;&gt;rump&lt;/span&gt;/../&lt;span class=&quot;hljs-title&quot;&gt;kern&lt;/span&gt;/&lt;span class=&quot;hljs-title&quot;&gt;sys_generic&lt;/span&gt;.&lt;span class=&quot;hljs-title&quot;&gt;c&lt;/span&gt;:671&lt;/span&gt;
2454 &lt;span class=&quot;hljs-comment&quot;&gt;#2 0x000075b0de6b8957 in sy_call (rval=0x7f7fffccd810, uap=0x7f7fffccd820, l=0x75b0de126500, &lt;/span&gt;
2455 sy=&amp;lt;optimized &lt;span class=&quot;hljs-keyword&quot;&gt;out&lt;/span&gt;&amp;gt;) at /usr/src/&lt;span class=&quot;hljs-class&quot;&gt;&lt;span class=&quot;hljs-keyword&quot;&gt;lib&lt;/span&gt;/&lt;span class=&quot;hljs-title&quot;&gt;librump&lt;/span&gt;/../../&lt;span class=&quot;hljs-title&quot;&gt;sys&lt;/span&gt;/&lt;span class=&quot;hljs-title&quot;&gt;rump&lt;/span&gt;/../&lt;span class=&quot;hljs-title&quot;&gt;sys&lt;/span&gt;/&lt;span class=&quot;hljs-title&quot;&gt;syscallvar&lt;/span&gt;.&lt;span class=&quot;hljs-title&quot;&gt;h&lt;/span&gt;:65&lt;/span&gt;
2456 &lt;span class=&quot;hljs-comment&quot;&gt;#3 sy_invoke (code=54, rval=0x7f7fffccd810, uap=0x7f7fffccd820, l=0x75b0de126500, sy=&amp;lt;optimized out&amp;gt;)&lt;/span&gt;
2457 at /usr/src/&lt;span class=&quot;hljs-class&quot;&gt;&lt;span class=&quot;hljs-keyword&quot;&gt;lib&lt;/span&gt;/&lt;span class=&quot;hljs-title&quot;&gt;librump&lt;/span&gt;/../../&lt;span class=&quot;hljs-title&quot;&gt;sys&lt;/span&gt;/&lt;span class=&quot;hljs-title&quot;&gt;rump&lt;/span&gt;/../&lt;span class=&quot;hljs-title&quot;&gt;sys&lt;/span&gt;/&lt;span class=&quot;hljs-title&quot;&gt;syscallvar&lt;/span&gt;.&lt;span class=&quot;hljs-title&quot;&gt;h&lt;/span&gt;:94&lt;/span&gt;
2458 &lt;span class=&quot;hljs-comment&quot;&gt;#4 rump_syscall (num=num@entry=54, data=data@entry=0x7f7fffccd820, dlen=dlen@entry=24, &lt;/span&gt;
2459 retval=retval@entry=&lt;span class=&quot;hljs-number&quot;&gt;0x7f7fffccd810&lt;/span&gt;)
2460 at /usr/src/&lt;span class=&quot;hljs-class&quot;&gt;&lt;span class=&quot;hljs-keyword&quot;&gt;lib&lt;/span&gt;/&lt;span class=&quot;hljs-title&quot;&gt;librump&lt;/span&gt;/../../&lt;span class=&quot;hljs-title&quot;&gt;sys&lt;/span&gt;/&lt;span class=&quot;hljs-title&quot;&gt;rump&lt;/span&gt;/&lt;span class=&quot;hljs-title&quot;&gt;librump&lt;/span&gt;/&lt;span class=&quot;hljs-title&quot;&gt;rumpkern&lt;/span&gt;/&lt;span class=&quot;hljs-title&quot;&gt;rump&lt;/span&gt;.&lt;span class=&quot;hljs-title&quot;&gt;c&lt;/span&gt;:769&lt;/span&gt;
2461 &lt;span class=&quot;hljs-comment&quot;&gt;#5 0x000075b0de6ad2ca in rump___sysimpl_ioctl (fd=&amp;lt;optimized out&amp;gt;, com=&amp;lt;optimized out&amp;gt;, &lt;/span&gt;
2462 data=&amp;lt;optimized &lt;span class=&quot;hljs-keyword&quot;&gt;out&lt;/span&gt;&amp;gt;) at /usr/src/&lt;span class=&quot;hljs-class&quot;&gt;&lt;span class=&quot;hljs-keyword&quot;&gt;lib&lt;/span&gt;/&lt;span class=&quot;hljs-title&quot;&gt;librump&lt;/span&gt;/../../&lt;span class=&quot;hljs-title&quot;&gt;sys&lt;/span&gt;/&lt;span class=&quot;hljs-title&quot;&gt;rump&lt;/span&gt;/&lt;span class=&quot;hljs-title&quot;&gt;librump&lt;/span&gt;/&lt;span class=&quot;hljs-title&quot;&gt;rumpkern&lt;/span&gt;/&lt;span class=&quot;hljs-title&quot;&gt;rump_syscalls&lt;/span&gt;.&lt;span class=&quot;hljs-title&quot;&gt;c&lt;/span&gt;:979&lt;/span&gt;
2463 &lt;span class=&quot;hljs-comment&quot;&gt;#6 0x0000000000400bf7 in main (argc=1, argv=0x7f7fffccd8c8) at test.c:15&lt;/span&gt;
2464 &lt;/code&gt;&lt;/pre&gt;
2465 &lt;p&gt;in the rump using a fuzzer that uses pip2, dup2 and ioctl syscalls and specific arguments that are known to cause a crash upon which my work develops.&lt;/p&gt;
2466 &lt;p&gt;&lt;a href=&quot;https://github.com/adityavardhanpadala/rumpsyscallfuzz/blob/master/honggfuzz/ioctl/ioctl_fuzz2.c&quot;&gt;https://github.com/adityavardhanpadala/rumpsyscallfuzz/blob/master/honggfuzz/ioctl/ioctl_fuzz2.c&lt;/a&gt;&lt;/p&gt;
2467 &lt;p&gt;Since rump is a multithreaded process. Crash occurs in any of those threads. By using a core dump we can quickly investigate the crash and fetch the backtrace from gdb for verification however this is not viable in the long run as you would be loading your working directory with lots of core dumps which consume a lot of space. So we need a better way to reproduce crashes.&lt;/p&gt;
2468 &lt;h2 id=&quot;crash-reproducers&quot;&gt;Crash Reproducers&lt;/h2&gt;
2469 &lt;p&gt;Getting crash reproducers working took quite a while. If we look at HF_ITER() function in honggfuzz, it is a simple wrapper for HonggfuzzFetchData() to fetch buffer of fixed size from the fuzzer.&lt;/p&gt;
2470 &lt;pre&gt;&lt;code class=&quot;lang-cpp=&quot;&gt;void HonggfuzzFetchData(const uint8&lt;span class=&quot;hljs-emphasis&quot;&gt;_t** buf_&lt;/span&gt;ptr, size&lt;span class=&quot;hljs-emphasis&quot;&gt;_t* len_&lt;/span&gt;ptr) {
2471 &lt;span class=&quot;hljs-bullet&quot;&gt;.
2472 &lt;/span&gt;&lt;span class=&quot;hljs-bullet&quot;&gt;.
2473 &lt;/span&gt;&lt;span class=&quot;hljs-bullet&quot;&gt;.
2474 &lt;/span&gt;&lt;span class=&quot;hljs-bullet&quot;&gt;.
2475 &lt;/span&gt;&lt;span class=&quot;hljs-strong&quot;&gt;*buf_ptr = inputFile;
2476 *&lt;/span&gt;len&lt;span class=&quot;hljs-emphasis&quot;&gt;_ptr = (size_&lt;/span&gt;t)rcvLen;
2477 &lt;span class=&quot;hljs-bullet&quot;&gt;.
2478 &lt;/span&gt;&lt;span class=&quot;hljs-bullet&quot;&gt;.
2479 &lt;/span&gt;}
2480 &lt;/code&gt;&lt;/pre&gt;
2481 &lt;p&gt;And if we observe the attribute we notice that &lt;code&gt;inputFile&lt;/code&gt; is mmaped. &lt;/p&gt;
2482 &lt;pre&gt;&lt;code class=&quot;lang-cpp=&quot;&gt;&lt;span class=&quot;hljs-comment&quot;&gt;//libhfuzz/fetch.c:26&lt;/span&gt;
2483 &lt;span class=&quot;hljs-keyword&quot;&gt;if&lt;/span&gt; ((inputFile = mmap(&lt;span class=&quot;hljs-literal&quot;&gt;NULL&lt;/span&gt;, _HF_INPUT_MAX_SIZE, PROT_READ, MAP_SHARED, _HF_INPUT_FD, &lt;span class=&quot;hljs-number&quot;&gt;0&lt;/span&gt;)) ==
2484 MAP_FAILED) {
2485 PLOG_F(&lt;span class=&quot;hljs-string&quot;&gt;&quot;mmap(fd=%d, size=%zu) of the input file failed&quot;&lt;/span&gt;, _HF_INPUT_FD,
2486 (&lt;span class=&quot;hljs-keyword&quot;&gt;size_t&lt;/span&gt;)_HF_INPUT_MAX_SIZE);
2487 }
2488 &lt;/code&gt;&lt;/pre&gt;
2489 &lt;p&gt;So in a similar approach HF_ITER() can be modified to read input from a file and be mmapped so that we can reuse the reproducers generated by honggfuzz.&lt;/p&gt;
2490 &lt;p&gt;Attempts have been made to use getchar(3) for fetching the buffer via STDIN but for some unknown reason it failed so we switched to mmap(2) &lt;/p&gt;
2491 &lt;p&gt;So we overload HF_ITER() function whenever we require to reproduce a crash. I chose the following approach to use the reproducers. So whenever we need to reproduce a crash we just define CRASH_REPR.&lt;/p&gt;
2492 &lt;pre&gt;&lt;code class=&quot;lang-cpp=&quot;&gt;
2493 &lt;span class=&quot;hljs-function&quot;&gt;&lt;span class=&quot;hljs-keyword&quot;&gt;static&lt;/span&gt;
2494 &lt;span class=&quot;hljs-keyword&quot;&gt;void&lt;/span&gt; &lt;span class=&quot;hljs-title&quot;&gt;Initialize&lt;/span&gt;&lt;span class=&quot;hljs-params&quot;&gt;(&lt;span class=&quot;hljs-keyword&quot;&gt;void&lt;/span&gt;)&lt;/span&gt;
2495 &lt;/span&gt;{
2496 &lt;span class=&quot;hljs-meta&quot;&gt;#&lt;span class=&quot;hljs-meta-keyword&quot;&gt;ifdef&lt;/span&gt; CRASH_REPR&lt;/span&gt;
2497 FILE *fp = fopen(argv[&lt;span class=&quot;hljs-number&quot;&gt;1&lt;/span&gt;], &lt;span class=&quot;hljs-string&quot;&gt;&quot;r+&quot;&lt;/span&gt;);
2498 data = &lt;span class=&quot;hljs-built_in&quot;&gt;malloc&lt;/span&gt;(max_size);
2499 fread(data, max_size, &lt;span class=&quot;hljs-number&quot;&gt;1&lt;/span&gt;, fp);
2500 fclose(fp);
2501 &lt;span class=&quot;hljs-meta&quot;&gt;#&lt;span class=&quot;hljs-meta-keyword&quot;&gt;endif&lt;/span&gt;&lt;/span&gt;
2502 &lt;span class=&quot;hljs-comment&quot;&gt;// Initialise the rumpkernel only once.&lt;/span&gt;
2503 &lt;span class=&quot;hljs-keyword&quot;&gt;if&lt;/span&gt;(rump_init() != &lt;span class=&quot;hljs-number&quot;&gt;0&lt;/span&gt;)
2504 __builtin_trap();
2505 }
2506
2507 &lt;span class=&quot;hljs-meta&quot;&gt;#&lt;span class=&quot;hljs-meta-keyword&quot;&gt;ifdef&lt;/span&gt; CRASH_REPR&lt;/span&gt;
2508 &lt;span class=&quot;hljs-function&quot;&gt;&lt;span class=&quot;hljs-keyword&quot;&gt;void&lt;/span&gt; &lt;span class=&quot;hljs-title&quot;&gt;HF_ITER&lt;/span&gt;&lt;span class=&quot;hljs-params&quot;&gt;(&lt;span class=&quot;hljs-keyword&quot;&gt;uint8_t&lt;/span&gt; **buf, &lt;span class=&quot;hljs-keyword&quot;&gt;size_t&lt;/span&gt; *len)&lt;/span&gt; &lt;/span&gt;{
2509 *buf = (&lt;span class=&quot;hljs-keyword&quot;&gt;uint8_t&lt;/span&gt; *)data;
2510 *len = max_size;
2511 &lt;span class=&quot;hljs-keyword&quot;&gt;return&lt;/span&gt;;
2512 }
2513 &lt;span class=&quot;hljs-meta&quot;&gt;#&lt;span class=&quot;hljs-meta-keyword&quot;&gt;else&lt;/span&gt;&lt;/span&gt;
2514 &lt;span class=&quot;hljs-function&quot;&gt;EXTERN &lt;span class=&quot;hljs-keyword&quot;&gt;void&lt;/span&gt; &lt;span class=&quot;hljs-title&quot;&gt;HF_ITER&lt;/span&gt;&lt;span class=&quot;hljs-params&quot;&gt;(&lt;span class=&quot;hljs-keyword&quot;&gt;uint8_t&lt;/span&gt; **buf, &lt;span class=&quot;hljs-keyword&quot;&gt;size_t&lt;/span&gt; *len)&lt;/span&gt;&lt;/span&gt;;
2515 &lt;span class=&quot;hljs-meta&quot;&gt;#&lt;span class=&quot;hljs-meta-keyword&quot;&gt;endif&lt;/span&gt;&lt;/span&gt;
2516 &lt;/code&gt;&lt;/pre&gt;
2517 &lt;p&gt;This way we can easily reproduce crashes that we get and get the backtraces.&lt;/p&gt;
2518 &lt;h2 id=&quot;generating-c-reproducers&quot;&gt;Generating C reproducers&lt;/h2&gt;
2519 &lt;p&gt;Now the main goal is to create a c file which can reproduce the same crash occuring due to the reproducer. This is done by writing all the syscall executions to a file with arguments so they can directly be compiled and used.&lt;/p&gt;
2520 &lt;pre&gt;&lt;code class=&quot;lang-cpp=&quot;&gt;#ifdef CRASH_REPR
2521 FILE *fp = fopen(&lt;span class=&quot;hljs-string&quot;&gt;&quot;/tmp/repro.c&quot;&lt;/span&gt;,&lt;span class=&quot;hljs-string&quot;&gt;&quot;a+&quot;&lt;/span&gt;)&lt;span class=&quot;hljs-comment&quot;&gt;;&lt;/span&gt;
2522 fprintf(&lt;span class=&quot;hljs-name&quot;&gt;fp&lt;/span&gt;, &lt;span class=&quot;hljs-string&quot;&gt;&quot;rump_sys_ioctl(%&quot;&lt;/span&gt; PRIu8 &lt;span class=&quot;hljs-string&quot;&gt;&quot;, %&quot;&lt;/span&gt; PRIu64 &lt;span class=&quot;hljs-string&quot;&gt;&quot;);\n&quot;&lt;/span&gt;,get_u8(),get_ioctl_request())&lt;span class=&quot;hljs-comment&quot;&gt;;&lt;/span&gt;
2523 fclose(&lt;span class=&quot;hljs-name&quot;&gt;fp&lt;/span&gt;)&lt;span class=&quot;hljs-comment&quot;&gt;;&lt;/span&gt;
2524 #else
2525 rump_sys_ioctl(&lt;span class=&quot;hljs-name&quot;&gt;get_u8&lt;/span&gt;(), get_ioctl_request())&lt;span class=&quot;hljs-comment&quot;&gt;;&lt;/span&gt;
2526 #endif
2527 &lt;/code&gt;&lt;/pre&gt;
2528 &lt;p&gt;I followed the same above method for all the syscalls that are executed. So I get a proper order of syscalls executed in native c code that I can simply reuse.&lt;/p&gt;
2529 &lt;h3 id=&quot;obstacles&quot;&gt;Obstacles&lt;/h3&gt;
2530 &lt;p&gt;The number of times each syscall is executed before getting to a crash is quite high. So trying to perform a write to a file or STDOUT will create a lot of overhead when the number of syscalls executed are quite high. This method is good enough but a bit of optimization will make it even better.&lt;/p&gt;
2531 &lt;h2 id=&quot;to-do&quot;&gt;To-Do&lt;/h2&gt;
2532 &lt;ul&gt;
2533 &lt;li&gt;./build.sh building rump on linux+netbsd&lt;/li&gt;
2534 &lt;li&gt;pregenerating fuzzer input using the implementation similar to that used in syzkaller.&lt;/li&gt;
2535 &lt;/ul&gt;
2536 &lt;p&gt;Finally I thank my mentors Siddharth Muralee, Maciej Grochowski, Christos Zoulas for their guidance and Kamil Rytarowski for his constant support whenever I needed it.&lt;/p&gt;</content>
2537 </entry>
2538 <entry>
2539 <id>https://blog.netbsd.org/tnf/entry/gsoc_reports_enhancing_syzkaller_support1</id>
2540 <title type="html">GSoC Reports: Enhancing Syzkaller support for NetBSD, Part 2</title>
2541 <author><name>Kamil Rytarowski</name></author>
2542 <link rel="alternate" type="text/html" href="https://blog.netbsd.org/tnf/entry/gsoc_reports_enhancing_syzkaller_support1"/>
2543 <published>2020-08-05T08:10:10+00:00</published>
2544 <updated>2020-08-05T08:10:10+00:00</updated>
2545 <category term="/The NetBSD Foundation" label="The NetBSD Foundation" />
2546 <category term="gsoc" scheme="http://roller.apache.org/ns/tags/" />
2547 <category term="syzkaller" scheme="http://roller.apache.org/ns/tags/" />
2548 <category term="fuzzing" scheme="http://roller.apache.org/ns/tags/" />
2549 <summary type="html">&lt;i&gt;This report was prepared by Ayushi Sharma as a part of Google Summer of Code 2020&lt;/i&gt;
2550
2551 &lt;p&gt;As a part of Google summer code 2020, I have been working on &lt;a href=&quot;https://wiki.netbsd.org/projects/project/syzkaller/&quot;&gt;Enhance the Syzkaller support for NetBSD&lt;/a&gt;. This post summarises the work done in the past month.&lt;/p&gt;
2552
2553 &lt;p&gt;For work done in the first coding period, you can take a look at the &lt;a href=&quot;https://blog.netbsd.org/tnf/entry/gsoc_reports_enhancing_syzkaller_support&quot;&gt;previous post&lt;/a&gt;.&lt;/p&gt;
2554
2555 &lt;h2&gt;Automation for enhancement&lt;/h2&gt;
2556 &lt;p&gt;With an aim of increasing the number of syscalls fuzzed, we have decided to automate the addition of descriptions for syscalls as well as ioctl device drivers in a customised way for NetBSD.&lt;/p&gt;</summary>
2557 <content type="html">&lt;i&gt;This report was prepared by Ayushi Sharma as a part of Google Summer of Code 2020&lt;/i&gt;
2558
2559 &lt;p&gt;As a part of Google summer code 2020, I have been working on &lt;a href=&quot;https://wiki.netbsd.org/projects/project/syzkaller/&quot;&gt;Enhance the Syzkaller support for NetBSD&lt;/a&gt;. This post summarises the work done in the past month.&lt;/p&gt;
2560
2561 &lt;p&gt;For work done in the first coding period, you can take a look at the &lt;a href=&quot;https://blog.netbsd.org/tnf/entry/gsoc_reports_enhancing_syzkaller_support&quot;&gt;previous post&lt;/a&gt;.&lt;/p&gt;
2562
2563 &lt;h2&gt;Automation for enhancement&lt;/h2&gt;
2564 &lt;p&gt;With an aim of increasing the number of syscalls fuzzed, we have decided to automate the addition of descriptions for syscalls as well as ioctl device drivers in a customised way for NetBSD.&lt;/p&gt;
2565
2566 &lt;h2&gt;Design&lt;/h2&gt;
2567 &lt;p&gt;All the ioctl commands for a device driver in NetBSD are stored inside the &lt;q&gt;/src/sys/dev/&amp;ltdriver_name&amp;gt/&lt;/q&gt; folder. The idea is to get information related to a particular ioctl command by extracting required information from the source code of drivers. To achieve the same, we have broken down our project into majorly three phases.&lt;/p&gt;
2568 &lt;ol&gt;
2569 &lt;li&gt;Generating preprocessed files&lt;/li&gt;
2570 &lt;li&gt;Extracting information required for generating descriptions&lt;/li&gt;
2571 &lt;li&gt;Conversion to syzkaller&#8217;s grammar&lt;/li&gt;
2572 &lt;/ol&gt;
2573
2574 &lt;img src=&quot;//netbsd.org/~kamil/gsoc_2020/sys2syz.png&quot; alt=&quot;Design&quot; style=&quot;border:10px solid white;&quot;&gt;
2575
2576
2577 &lt;h2&gt;Generating Preprocessed files&lt;/h2&gt;
2578 &lt;p&gt;For a given preprocessed file, c2xml tool outputs the preprocessed C code in xml format. Further, the intermediate xml format descriptions would help us to smoothly transform the c code to syzkaller specific descriptions, in the last stage of this tool. We have used &lt;a href=https://github.com/rizsotto/Bear&gt;Bear&lt;/a&gt; as an aid for fetching commands to preprocess files for a particular driver. Bear generates a file called compile_commands.json which stores the commands used for compiling a file in json format. We then run these commands with &#8216;-E&#8217; gcc flag to fetch the preprocessed files.These preprocessed files then serve as an input to the c2xml program.&lt;/p&gt;
2579
2580
2581 &lt;h2&gt;Extractor&lt;/h2&gt;
2582 &lt;p&gt;Definition of &lt;a href=&quot;//man.NetBSD.org/ioctl.9&quot;&gt;ioctl&lt;/a&gt; calls defined in header files of device driver in NetBSD can be broken down to:&lt;/p&gt;
2583
2584 &lt;img src=&quot;//netbsd.org/~kamil/gsoc_2020/ioctl_def.png&quot; alt=&quot;ioctl&quot; &gt;
2585
2586 &lt;p&gt;When we see it from syzkaller&#8217;s perspective, there are basically three significant parts we need to extract for adding description to syzkaller.&lt;/p&gt;
2587
2588 &lt;p&gt;Description of a particular ioctl command acc to syzkaller&#8217;s grammar:&lt;/p&gt;
2589
2590 &lt;p&gt;ioctl$FOOIOCTL(fd &amp;ltfd_driver&amp;gt, cmd const[FOOIOCTL], pt ptr[DIR, &amp;ltptr_type&amp;gt])
2591
2592 &lt;br&gt;
2593 &lt;img src=&quot;//netbsd.org/~kamil/gsoc_2020/ioctl_desc_1.png&quot; alt=&quot;ioctl description&quot;&gt;
2594 &lt;br&gt;
2595 &lt;img src=&quot;//netbsd.org/~kamil/gsoc_2020/ioctl_desc_2.png&quot; alt=&quot;ioctl description&quot;&gt;
2596
2597 &lt;p&gt;These definitions can be grepped from a device&#8217;s header files. The type information or description for pointer can then be extracted from the output files generated by c2xml. If the third argument is a struct, the direction of the pointer is determined with the help of fun() macros.
2598
2599
2600 &lt;h2&gt;To-Do&lt;/h2&gt;
2601 &lt;p&gt;The extracted descriptions have to be converted into syzkaller-friendly grammer. We plan to add support for syscalls too , which would ease the addition of complex compat syscalls. This would help us to increase the syzkaller&#8217;s coverage significantly.
2602
2603 &lt;h2&gt;Stats&lt;/h2&gt;
2604 Along with this, We have continued to add support for few more syscalls these include:
2605 &lt;ul&gt;
2606 &lt;li&gt;ksem(2) family&lt;/li&gt;
2607 &lt;li&gt;mount(2) family&lt;/li&gt;
2608 &lt;/ul&gt;
2609 Syscalls related to sockets have also been added. This has increased syscall coverage percentage to 50.35.
2610
2611 &lt;p&gt;Atlast, I would like to thank my mentors - Cryo, Siddharth Muralee and Santhosh along with Kamil for their guidance and support. I am thankful to NetBSD community too along with Google for providing me such an amazing opportunity. </content>
2612 </entry>
2613 <entry>
2614 <id>https://blog.netbsd.org/tnf/entry/the_gnu_gdb_debugger_and2</id>
2615 <title type="html">The GNU GDB Debugger and NetBSD (Part 3)</title>
2616 <author><name>Kamil Rytarowski</name></author>
2617 <link rel="alternate" type="text/html" href="https://blog.netbsd.org/tnf/entry/the_gnu_gdb_debugger_and2"/>
2618 <published>2020-08-04T16:45:37+00:00</published>
2619 <updated>2020-08-04T16:45:37+00:00</updated>
2620 <category term="/The NetBSD Foundation" label="The NetBSD Foundation" />
2621 <category term="gdb" scheme="http://roller.apache.org/ns/tags/" />
2622 <summary type="html">The NetBSD team of developers maintains two copies of GDB:
2623 &lt;ul&gt;
2624 &lt;li&gt;One in the base-system with a stack of local patches.&lt;/li&gt;
2625 &lt;li&gt;One in pkgsrc with mostly build fix patches.&lt;/li&gt;
2626 &lt;/ul&gt;
2627 &lt;p&gt;
2628 The base-system version of GDB (GPLv3) still relies on a set of local patches.
2629 I set a goal to reduce the local patches to bare minimum, ideally reaching no local modifications at all.</summary>
2630 <content type="html">The NetBSD team of developers maintains two copies of GDB:
2631 &lt;ul&gt;
2632 &lt;li&gt;One in the base-system with a stack of local patches.&lt;/li&gt;
2633 &lt;li&gt;One in pkgsrc with mostly build fix patches.&lt;/li&gt;
2634 &lt;/ul&gt;
2635 &lt;p&gt;
2636 The base-system version of GDB (GPLv3) still relies on a set of local patches.
2637 I set a goal to reduce the local patches to bare minimum, ideally reaching no local modifications at all.
2638 &lt;p&gt;
2639 &lt;h1&gt;GDB changes&lt;/h1&gt;
2640 &lt;p&gt;
2641 I&apos;ve written an integration of GDB with fork(2) and vfork(2) events.
2642 Unfortunately, this support (present in a local copy of GDB in the base-system) had not been merged so far, because
2643 there is a generic kernel regression with the pg_jobc variable. This variable can be called
2644 a reference counter of the number of processes within a process group that has a parent with control over a terminal.
2645 The semantics of this variable are not very well defined and in the result the number can become negative.
2646 This unexpected state of pg_jobc resulted in spurious crashes during kernel fuzzing. As a result
2647 new kernel assertions checking for non-negative pg_jobc values were introduced in order to catch the anomalies quickly.
2648 GDB as a ptrace(2)-based application happened to reproduce negative pg_jobc values quickly and reliably and this stopped
2649 the further adoption of the fork(2) and vfork(2) patch in GDB, until the pg_jobc behavior is enhanced.
2650 I was planning to include support for posix_spawn(3) events as well, as they are implemented as a first-class
2651 operation through a syscall, however this is also blocked by the pg_jobc blocker.
2652 &lt;p&gt;
2653 A local patch for GDB is stored
2654 &lt;a href=&quot;//netbsd.org/~kamil/patch-00276-gdb-fork-vfork-events.txt&quot;&gt;here&lt;/a&gt;
2655 for the time being.
2656 &lt;p&gt;
2657 I&apos;ve enable multi-process mode in the NetBSD native target.
2658 This enabled proper support for multiple inferiors and ptrace(2)
2659 assisted management of the inferior processes and their threads.
2660 &lt;p&gt;
2661 &lt;pre&gt;
2662 (gdb) info inferior
2663 Num Description Connection Executable
2664 * 1 process 14952 1 (native) /usr/bin/dig
2665 2 &amp;lt;null&amp;gt; 1 (native)
2666 3 process 25684 1 (native) /bin/ls
2667 4 &amp;lt;null&amp;gt; 1 (native) /bin/ls
2668 &lt;/pre&gt;
2669 &lt;p&gt;
2670 Without this change, additional inferiors could be already added, but not properly controlled.
2671 &lt;p&gt;
2672 I&apos;ve implemented the xfer_partial TARGET_OBJECT_SIGNAL_INFO support for NetBSD.
2673 NetBSD implements reading and overwriting siginfo_t received by the
2674 tracee. With TARGET_OBJECT_SIGNAL_INFO signal information can be
2675 examined and modified through the special variable $_siginfo.
2676 Currently NetBSD uses an identical siginfo type on
2677 all architectures, so there is no support for architecture-specific fields.
2678 &lt;pre&gt;
2679 (gdb) b main
2680 Breakpoint 1 at 0x71a0
2681 (gdb) r
2682 Starting program: /bin/ps
2683
2684 Breakpoint 1, 0x00000000002071a0 in main ()
2685 (gdb) p $_siginfo
2686 $1 = {si_pad = {5, 0, 0, 0, 1, 0 &lt;repeats 19 times&gt;, 1, 0 &lt;repeats 103 times&gt;}, _info = {_signo = 5,
2687 _code = 1, _errno = 0, _pad = 0, _reason = {_rt = {_pid = 0, _uid = 0, _value = {sival_int = 1,
2688 sival_ptr = 0x1}}, _child = {_pid = 0, _uid = 0, _status = 1, _utime = 0, _stime = 0},
2689 _fault = {_addr = 0x0, _trap = 1, _trap2 = 0, _trap3 = 0}, _poll = {_band = 0, _fd = 1},
2690 _syscall = {_sysnum = 0, _retval = {0, 1}, _error = 0, _args = {0, 0, 0, 0, 0, 0, 0, 0}},
2691 _ptrace_state = {_pe_report_event = 0, _option = {_pe_other_pid = 0, _pe_lwp = 0}}}}}
2692 &lt;/pre&gt;
2693 &lt;p&gt;
2694 NetBSD, contrary to Linux and other BSDs, supports a ptrace(2) operation to
2695 generate a core(5) file from a running process. This operation is used in the
2696 base-system gcore(1) program. The gcore functionality is also delivered by GDB, and
2697 I have prepared new code for GDB to wire PT_DUMPCORE into the GDB code for NetBSD,
2698 and thus support GDB&apos;s gcore functionality.
2699 This patch is still waiting in upstream review.
2700 A local copy of the patch is
2701 &lt;a href=&quot;//netbsd.org/~kamil/patch-00277-gdb-dumpcore.txt&quot;&gt;here&lt;/a&gt;.
2702 &lt;p&gt;
2703 &lt;pre&gt;
2704 (gdb) r
2705 Starting program: /bin/ps
2706
2707 Breakpoint 1, 0x00000000002071a0 in main ()
2708 (gdb) gcore
2709 Saved corefile core.4378
2710 (gdb) !file core.4378
2711 core.4378: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), NetBSD-style, from &apos;ps&apos;, pid=4378, uid=1000, gid=100, nlwps=1, lwp=4378 (signal 5/code 1)
2712 &lt;/pre&gt;
2713 &lt;p&gt;
2714 &lt;h1&gt;Plan for the next milestone&lt;/h1&gt;
2715 &lt;p&gt;
2716 Rewrite the gdbserver support and submit upstream.
2717 </content>
2718 </entry>
2719 <entry>
2720 <id>https://blog.netbsd.org/tnf/entry/gsoc_reports_benchmarking_netbsd_first</id>
2721 <title type="html">GSoC Reports: Benchmarking NetBSD, first evaluation report</title>
2722 <author><name>Leonardo Taccari</name></author>
2723 <link rel="alternate" type="text/html" href="https://blog.netbsd.org/tnf/entry/gsoc_reports_benchmarking_netbsd_first"/>
2724 <published>2020-07-16T09:38:58+00:00</published>
2725 <updated>2020-07-17T16:00:11+00:00</updated>
2726 <category term="/General" label="General" />
2727 <category term="gsoc" scheme="http://roller.apache.org/ns/tags/" />
2728 <category term="benchmarking" scheme="http://roller.apache.org/ns/tags/" />
2729 <category term="pkgsrc" scheme="http://roller.apache.org/ns/tags/" />
2730 <summary type="html">&lt;p&gt;This report was written by Apurva Nandan as part of Google Summer of Code 2020.&lt;/p&gt;
2731
2732 &lt;p&gt;My GSoC project under NetBSD involves developing an automated
2733 regression and performance test framework for NetBSD that offers
2734 reproducible benchmarking results with detailed history and logs across
2735 various hardware &amp;amp; architectures.&lt;/p&gt;
2736
2737 &lt;p&gt;To achieve this performance testing framework, I am using the
2738 &lt;a href=&quot;https://www.phoronix-test-suite.com/&quot;&gt;Phoronix Test Suite (PTS)&lt;/a&gt;
2739 which is an open-source, cross-platform automated testing/benchmarking
2740 software for Linux, Windows and BSD environments. It allows the
2741 creation of new tests using simple XML files and shell scripts and
2742 integrates with revision control systems for per-commit regression
2743 testing.&lt;/p&gt;</summary>
2744 <content type="html">&lt;p&gt;This report was written by Apurva Nandan as part of Google Summer of Code 2020.&lt;/p&gt;
2745
2746 &lt;p&gt;My GSoC project under NetBSD involves developing an automated
2747 regression and performance test framework for NetBSD that offers
2748 reproducible benchmarking results with detailed history and logs across
2749 various hardware &amp;amp; architectures.&lt;/p&gt;
2750
2751 &lt;p&gt;To achieve this performance testing framework, I am using the
2752 &lt;a href=&quot;https://www.phoronix-test-suite.com/&quot;&gt;Phoronix Test Suite (PTS)&lt;/a&gt;
2753 which is an open-source, cross-platform automated testing/benchmarking
2754 software for Linux, Windows and BSD environments. It allows the
2755 creation of new tests using simple XML files and shell scripts and
2756 integrates with revision control systems for per-commit regression
2757 testing.&lt;/p&gt;
2758
2759 &lt;h2&gt;About PTS&lt;/h2&gt;
2760
2761 &lt;h3&gt;PTS core&lt;/h3&gt;
2762
2763 &lt;p&gt;PTS core is the engine of the Phoronix Test Suite and handles the following jobs:&lt;/p&gt;
2764
2765 &lt;ul&gt;
2766 &lt;li&gt;Regularly updating the test profiles, test suites, and repository index.&lt;/li&gt;
2767 &lt;li&gt;Downloading, compilation, installation and evaluation of test packages.&lt;/li&gt;
2768 &lt;li&gt;Test result management and uploading results and user-defined
2769 test-profiles/suites to OpenBenchmarking.org&lt;/li&gt;
2770 &lt;/ul&gt;
2771
2772
2773 &lt;h3&gt;Test-profiles/Suites &amp;amp; OpenBenchmarking&lt;/h3&gt;
2774
2775 &lt;p&gt;Test-profiles are light-weight XMLs and shell scripts that allow easy
2776 installation of the benchmarking packages and their evaluation.
2777 Test-profiles generally contains the following files:&lt;/p&gt;
2778
2779 &lt;ul&gt;
2780 &lt;li&gt;&lt;code&gt;downloads.xml&lt;/code&gt;: Stores the links of the benchmarking packages,
2781 required additional packages, patches to be applied, etc. with their
2782 hash sums.&lt;/li&gt;
2783 &lt;li&gt;&lt;code&gt;install.sh&lt;/code&gt;: Actual compilation, installation, and test
2784 evaluation shell script. Also handles the task of applying patches.&lt;/li&gt;
2785 &lt;li&gt;&lt;code&gt;results-definition.xml&lt;/code&gt;: Meta-data to define the information for
2786 test result data (e.g. result data units, LIB or HIB, etc.)&lt;/li&gt;
2787 &lt;li&gt;&lt;code&gt;test-definition.xml&lt;/code&gt;: Meta-data to specify test-profile details,
2788 such as compatible OS, external dependencies, test arguments, etc.&lt;/li&gt;
2789 &lt;/ul&gt;
2790
2791
2792 &lt;p&gt;A simple test-profile
2793 &lt;a href=&quot;https://openbenchmarking.org/innhold/91db3ffff901d12dabd732bc568a44d02e5c6387&quot;&gt;C-Ray-1.2.0&lt;/a&gt;
2794 can be seen to get an idea of the XMLs and shell scripts.&lt;/p&gt;
2795
2796 &lt;p&gt;Test-suites are bundles of related test-profiles or more suites,
2797 focusing on a subsystem or certain category of tests e.g. Disk Test
2798 Suite, Kernel, CPU suite, etc.&lt;/p&gt;
2799
2800 &lt;p&gt;&lt;a href=&quot;https://openbenchmarking.org/&quot;&gt;OpenBenchmarking.org&lt;/a&gt; serves as a
2801 repository for storing test profiles, test suites, hence allowing
2802 new/updated tests to be seamlessly obtained via PTS.
2803 OpenBenchmarking.org also provides a platform to store the test result
2804 data openly and do a comparison between any test results stored in the
2805 OpenBenchmarking.org cloud.&lt;/p&gt;
2806
2807 &lt;h2&gt;Usage&lt;/h2&gt;
2808
2809 &lt;h3&gt;Test installation&lt;/h3&gt;
2810
2811 &lt;p&gt;The command:&lt;/p&gt;
2812
2813 &lt;p&gt;&lt;code&gt;$ phoronix-test-suite install test-profile-name&lt;/code&gt;&lt;/p&gt;
2814
2815 &lt;p&gt;Fetches the sources of the tests related to the the test-profile in
2816 &lt;code&gt;~/.phoronix-test-suite/installed-tests&lt;/code&gt;, applies patches and
2817 carries out compilation of test sources for generating the executables
2818 to run the tests.&lt;/p&gt;
2819
2820 &lt;h3&gt;Test execution&lt;/h3&gt;
2821
2822 &lt;p&gt;The command:&lt;/p&gt;
2823
2824 &lt;p&gt;&lt;code&gt;$ phoronix-test-suite run test-profile-name&lt;/code&gt;&lt;/p&gt;
2825
2826 &lt;p&gt;Performs installation of the test (similar to &lt;code&gt;$ phoronix-test-suite
2827 install test-profile-name&lt;/code&gt;) followed by exectution the binary from
2828 the compiled sources of the test in
2829 &lt;code&gt;~/.phoronix-test-suite/installed-tests&lt;/code&gt; and finally reports the
2830 tests results/outcome to the user and provides an option to upload
2831 results to OpenBenchmarking.org.&lt;/p&gt;
2832
2833 &lt;h2&gt;Progress in the first phase of GSoC&lt;/h2&gt;
2834
2835 &lt;h3&gt;pkgsrc-wip&lt;/h3&gt;
2836
2837 &lt;p&gt;The first task performed by me was upgrading the Phoronix Test Suite
2838 from version 8.8 to the latest stable version 9.6.1 in &lt;code&gt;pkgsrc-wip&lt;/code&gt;
2839 and is available as
2840 &lt;a href=&quot;https://pkgsrc.se/wip/phoronix-test-suite&quot;&gt;wip/phoronix-test-suite&lt;/a&gt;.
2841 You can have a look at the
2842 &lt;a href=&quot;https://github.com/phoronix-test-suite/phoronix-test-suite/blob/master/ChangeLog&quot;&gt;PTS Changelog&lt;/a&gt;
2843 to know about the improvements between these two versions.&lt;/p&gt;
2844
2845 &lt;h4&gt;Major Commits:&lt;/h4&gt;
2846
2847 &lt;ul&gt;
2848 &lt;li&gt;&lt;a href=&quot;https://github.com/NetBSD/pkgsrc-wip/commit/0073e18af14a97e33542a15d5a2940ed0b353897&quot;&gt;wip/phoronix-test-suite: update phoronix-test-suite to 9.6.1&lt;/a&gt;&lt;/li&gt;
2849 &lt;li&gt;&lt;a href=&quot;https://github.com/NetBSD/pkgsrc-wip/commit/959a9a0821fb155d11a2a68d2271233135375df1&quot;&gt;Added required PHP dependencies &amp;amp; removed TODO file&lt;/a&gt;&lt;/li&gt;
2850 &lt;/ul&gt;
2851
2852
2853 &lt;p&gt;Currently, the PTS upgrade to 9.6.1 is subject to more modifications
2854 and fixes before it gets finally merged to the &lt;code&gt;pkgsrc&lt;/code&gt; upstream.
2855 To test the Phoronix Test Suite 9.6.1 on NetBSD till then, you can
2856 setup &lt;code&gt;pkgsrc-wip&lt;/code&gt; on your system via:&lt;/p&gt;
2857
2858 &lt;pre&gt;
2859 $ cd /usr/pkgsrc
2860 $ git clone git://wip.pkgsrc.org/pkgsrc-wip.git wip
2861 &lt;/pre&gt;
2862
2863 &lt;p&gt;To learn more about pkgsrc-wip please see
2864 &lt;a href=&quot;https://pkgsrc.org/wip/&quot;&gt;The pkgsrc-wip project homepage&lt;/a&gt;.&lt;/p&gt;
2865
2866 &lt;p&gt;After setting up &lt;code&gt;pkgsrc-wip&lt;/code&gt;, use the following command for
2867 installation of PTS 9.6.1:&lt;/p&gt;
2868
2869 &lt;pre&gt;
2870 $ cd /usr/pkgsrc/wip/phoronix-test-suite
2871 $ make install
2872 &lt;/pre&gt;
2873
2874 &lt;p&gt;If any new build/installation errors are encountered, please document
2875 them in wip/phoronix-test-suite/TODO file and/or contact me.&lt;/p&gt;
2876
2877 &lt;h3&gt;Testing &amp;amp; Debugging&lt;/h3&gt;
2878
2879 &lt;p&gt;As we now know, having a look over OpenBenchmarking.org&amp;rsquo;s available
2880 test-profiles section or using
2881 &lt;code&gt;$ phoronix-test-suite list-available-tests&lt;/code&gt;,
2882 there are 309 test-profiles available on PTS for Linux, and only 166 of
2883 309 tests have been ported to NetBSD (can be differentiated by a
2884 thunder symbol on the test profile on the website). These 166
2885 test-profiles can be fetch, build and executed on NetBSD using just a
2886 single command &lt;code&gt;$ phoronix-test-suite run test-profile-name&lt;/code&gt;. I am
2887 ignoring the graphics ones in both Linux &amp;amp; NetBSD as GPU benchmarking
2888 wasn&amp;rsquo;t required in the project.&lt;/p&gt;
2889
2890 &lt;p&gt;Many of the test profiles available on NetBSD have installation, build,
2891 or runtime errors i.e. they don&amp;rsquo;t work out of the box using the command
2892 &lt;code&gt;$ phoronix-test-suite run test-profile-name&lt;/code&gt;. I ran 90 of these
2893 test profiles on a NetBSD-current x86_64 QEMU VM (took a lot of time
2894 due to the heavy tests) and have reported the status in the sheet:
2895 &lt;a href=&quot;https://drive.google.com/file/d/1gW76JI7W-Jpeczns49k1bBIVr8bb6QuX/view?usp=sharing&quot;&gt;NetBSD GSoC: Test Porting Progress Report&lt;/a&gt;&lt;/p&gt;
2896
2897 &lt;p&gt;You can have a look at how a PTS test-profile under action looks like!&lt;/p&gt;
2898
2899 &lt;h4&gt;Aircrack-NG test-profile demonstration&lt;/h4&gt;
2900
2901 &lt;p&gt;Aircrack-ng is a tool for assessing WiFi/WLAN network security.&lt;/p&gt;
2902
2903 &lt;p&gt;The PTS test-profile is available at:
2904 &lt;a href=&quot;https://openbenchmarking.org/test/pts/aircrack-ng&quot;&gt;https://openbenchmarking.org/test/pts/aircrack-ng&lt;/a&gt;&lt;/p&gt;
2905
2906 &lt;p&gt;&lt;img src=&quot;//www.NetBSD.org/~leot/blog-posts/imgs/pts_aircrack_ng_test.png&quot; alt=&quot;Screenshot of aircrack-ng PTS test-profile results&quot; width=&quot;800&quot; /&gt;&lt;/p&gt;
2907
2908 &lt;p&gt;For further information, complete results can be seen at
2909 &lt;a href=&quot;https://openbenchmarking.org/result/2007116-NI-AIRCRACKN51&quot;&gt;https://openbenchmarking.org/result/2007116-NI-AIRCRACKN51&lt;/a&gt;&lt;/p&gt;
2910
2911 &lt;h4&gt;AOBench test-profile demonstration&lt;/h4&gt;
2912
2913 &lt;p&gt;AOBench is a lightweight ambient occlusion renderer, written in C.&lt;/p&gt;
2914
2915 &lt;p&gt;The PTS test-profile is available at:
2916 &lt;a href=&quot;https://openbenchmarking.org/test/pts/aircrack-ng&quot;&gt;https://openbenchmarking.org/test/pts/aircrack-ng&lt;/a&gt;&lt;/p&gt;
2917
2918 &lt;p&gt;&lt;img src=&quot;//www.NetBSD.org/~leot/blog-posts/imgs/pts_aobench_test.png&quot; alt=&quot;Screenshot of aobench PTS test-profile results&quot; width=&quot;800&quot; /&gt;&lt;/p&gt;
2919
2920 &lt;p&gt;For further information, complete results can be seen at:
2921 &lt;a href=&quot;https://openbenchmarking.org/result/2007148-NI-AOBENCHTE94&quot;&gt;https://openbenchmarking.org/result/2007148-NI-AOBENCHTE94&lt;/a&gt;&lt;/p&gt;
2922
2923 &lt;h4&gt;Debugging and fixing non-working test-profiles&lt;/h4&gt;
2924
2925 &lt;p&gt;After testing these test-profiles, I debugged the following build
2926 errors in the following test-profiles:&lt;/p&gt;
2927
2928 &lt;ul&gt;
2929 &lt;li&gt;pts/t-test1-1.0.1: &lt;code&gt;memalign()&lt;/code&gt; not available on NetBSD&lt;/li&gt;
2930 &lt;li&gt;pts/coremark-1.0.0: calling bmake instead of gmake&lt;/li&gt;
2931 &lt;li&gt;pts/apache-siege-1.0.4: Missing &lt;code&gt;signal.h&lt;/code&gt; header&lt;/li&gt;
2932 &lt;li&gt;pts/mbw-1.0.0: &lt;code&gt;mempcpy()&lt;/code&gt; not available on NetBSD&lt;/li&gt;
2933 &lt;/ul&gt;
2934
2935
2936 &lt;p&gt;Fixes to the above bugs can be found in the sheet or in the GitHub
2937 repositories shared in the next paragraph.&lt;/p&gt;
2938
2939 &lt;p&gt;The modified test-profiles have been pushed to
2940 &lt;a href=&quot;https://github.com/apurvanandan1997/pts-test-profiles-dev&quot;&gt;pts-test-profiles-dev&lt;/a&gt;
2941 and the fix patches to
2942 &lt;a href=&quot;https://github.com/apurvanandan1997/pts-test-profiles-patches&quot;&gt;pts-test-profiles-patches&lt;/a&gt;.
2943 These patches are automatically downloaded by the debugged
2944 test-profiles and automatically applied before building of tests. The
2945 steps to install the debugged test-profiles have been added in the
2946 &lt;code&gt;README.md&lt;/code&gt; of
2947 &lt;a href=&quot;https://github.com/apurvanandan1997/pts-test-profiles-dev&quot;&gt;pts-test-profiles-dev&lt;/a&gt;.&lt;/p&gt;
2948
2949 &lt;p&gt;These patches have been added in the &lt;code&gt;downloads.xml&lt;/code&gt; of the
2950 modified test-profiles, and hence they get fetched during the test
2951 installation. The &lt;code&gt;install.sh&lt;/code&gt; script in these modified
2952 test-profiles applies them on the benchmarking packages if the
2953 operating system is detected as NetBSD, thereby removing the build
2954 errors.&lt;/p&gt;
2955
2956 &lt;h4&gt;coremark test-profile demonstration&lt;/h4&gt;
2957
2958 &lt;p&gt;You can have a look at the patched coremark-1.0.0 test-profile under
2959 execution:&lt;/p&gt;
2960
2961 &lt;p&gt;The patched PTS test-profile is available at:
2962 &lt;a href=&quot;https://github.com/apurvanandan1997/pts-test-profiles-dev/tree/master/coremark-1.0.0&quot;&gt;https://github.com/apurvanandan1997/pts-test-profiles-dev/tree/master/coremark-1.0.0&lt;/a&gt;&lt;/p&gt;
2963
2964 &lt;p&gt;This is a test of EEMBC CoreMark processor benchmark.&lt;/p&gt;
2965
2966 &lt;p&gt;&lt;img src=&quot;//www.NetBSD.org/~leot/blog-posts/imgs/pts_coremark_test.png&quot; alt=&quot;Screenshot of patched coremark PTS test-profile results&quot; width=&quot;800&quot; /&gt;&lt;/p&gt;
2967
2968 &lt;p&gt;For further information, complete results can be seen at:
2969 &lt;a href=&quot;https://openbenchmarking.org/result/2007148-NI-LOCALCORE64&quot;&gt;https://openbenchmarking.org/result/2007148-NI-LOCALCORE64&lt;/a&gt;&lt;/p&gt;
2970
2971 &lt;h4&gt;Porting more test-profiles to NetBSD&lt;/h4&gt;
2972
2973 &lt;p&gt;Attempts were made to port new test-profiles from Linux to NetBSD, but
2974 the major incompatibility issue was missing external dependencies in
2975 NetBSD. Hence, this may reduce the number of test-profiles we can
2976 actually port, but more sincere efforts will be made in this direction
2977 in the next phase.&lt;/p&gt;
2978
2979 &lt;h2&gt;Future Plans&lt;/h2&gt;
2980
2981 &lt;p&gt;My next steps would involve porting the non-available tests to NetBSD
2982 i.e. the non-graphics ones available on Linux but unavailable on
2983 NetBSD, and fix the remaining test-profiles for NetBSD.&lt;/p&gt;
2984
2985 &lt;p&gt;After gaining an availability of a decent number of test-profiles on
2986 NetBSD, I will use Phoromatic, a remote tests management system for the
2987 PTS (allows the automatic scheduling of tests, remote installation of
2988 new tests, and the management of multiple test systems) to host the
2989 live results of the automated benchmarking framework on
2990 benchmark.NetBSD.org, thereby delivering a functional performance and
2991 regression testing framework.&lt;/p&gt;
2992 </content>
2993 </entry>
2994 <entry>
2995 <id>https://blog.netbsd.org/tnf/entry/gsoc_reports_enhancing_syzkaller_support</id>
2996 <title type="html">GSoC Reports: Enhancing Syzkaller support for NetBSD, Part 1</title>
2997 <author><name>Kamil Rytarowski</name></author>
2998 <link rel="alternate" type="text/html" href="https://blog.netbsd.org/tnf/entry/gsoc_reports_enhancing_syzkaller_support"/>
2999 <published>2020-07-13T22:18:21+00:00</published>
3000 <updated>2020-07-13T22:18:21+00:00</updated>
3001 <category term="/The NetBSD Foundation" label="The NetBSD Foundation" />
3002 <category term="gsoc" scheme="http://roller.apache.org/ns/tags/" />
3003 <summary type="html">&lt;i&gt;This report was prepared by Ayushi Sharma as a part of Google Summer of Code 2020&lt;/i&gt;
3004
3005 &lt;p&gt;I have been working on the project - &lt;a href=&quot;https://wiki.netbsd.org/projects/project/syzkaller/&quot;&gt;Enhance the Syzkaller support for NetBSD&lt;/a&gt;, as a part of GSoc&#8217;20. Past two months have given me quite an enriching experience, pushing me to comprehend more knowledge on fuzzers. This post would give a peek into the work which has been done so far.&lt;/p&gt;</summary>
3006 <content type="html">&lt;i&gt;This report was prepared by Ayushi Sharma as a part of Google Summer of Code 2020&lt;/i&gt;
3007
3008 &lt;p&gt;I have been working on the project - &lt;a href=&quot;https://wiki.netbsd.org/projects/project/syzkaller/&quot;&gt;Enhance the Syzkaller support for NetBSD&lt;/a&gt;, as a part of GSoc&#8217;20. Past two months have given me quite an enriching experience, pushing me to comprehend more knowledge on fuzzers. This post would give a peek into the work which has been done so far.&lt;/p&gt;
3009
3010 &lt;h2&gt;Syzkaller&lt;/h2&gt;
3011 &lt;p&gt;Syzkaller is a coverage guided fuzzer developed by Google, to fuzz the system calls mainly keeping linux in mind. However, the support has been extended to fuzz the system calls of other Operating Systems as well for eg. Akaros, FreeBSD, NetBSD, OpenBSD, Windows etc.&lt;/p&gt;
3012
3013 &lt;p&gt; An automated system Syzbot continuously runs the syzkaller fuzzer on NetBSD kernel and reports the crashes &lt;p&gt;
3014
3015 &lt;img src=&quot;//netbsd.org/~kamil/gsoc_2020/syzbot.png&quot; alt=&quot;Syzbot&quot; height=&quot;333&quot;&gt;
3016
3017
3018 &lt;h2&gt;Increasing syscall support&lt;/h2&gt;
3019 &lt;p&gt;Initially, the syscall support for &lt;a href=&quot;https://gist.github.com/ais2397/1bed83535b7b65a7d488deb310a0dd78&quot;&gt;Linux&lt;/a&gt; as well as &lt;a href=&quot;https://gist.github.com/ais2397/b8886661c698be83fcc757858a2184b4&quot;&gt;FreeBSD&lt;/a&gt; was analysed by an automated script. Also &lt;a href=&quot;https://gist.github.com/ais2397/b6a737b9ad0836301194478549e733c8&quot;&gt;coverage of NetBSD syscalls&lt;/a&gt; was looked over. This helped us to easily port a few syscalls descriptions for NetBSD. The necessary tweaks were made according to the documentation which describes rules for writing syscall descriptions.&lt;/p&gt;
3020
3021 &lt;p&gt;Major groups of syscalls which have been added:
3022 &lt;ul&gt;
3023 &lt;li&gt;statfs&lt;/li&gt;
3024 &lt;li&gt;__getlogin
3025 &lt;li&gt;getsid&lt;/li&gt;
3026 &lt;li&gt;mknod&lt;/li&gt;
3027 &lt;li&gt;utimes&lt;/li&gt;
3028 &lt;li&gt;wait4&lt;/li&gt;
3029 &lt;li&gt;seek&lt;/li&gt;
3030 &lt;li&gt;setitimer&lt;/li&gt;
3031 &lt;li&gt;setpriority&lt;/li&gt;
3032 &lt;li&gt;getrusage&lt;/li&gt;
3033 &lt;li&gt;clock_settime&lt;/li&gt;
3034 &lt;li&gt;nanosleep&lt;/li&gt;
3035 &lt;li&gt;getdents &lt;/li&gt;
3036 &lt;li&gt;acct &lt;/li&gt;
3037 &lt;li&gt;dup&lt;/li&gt;
3038
3039 &lt;/ul&gt;
3040 &lt;/p&gt;
3041
3042 &lt;h2&gt;Bugs Found&lt;/h2&gt;
3043
3044 There were a few bugs reported as a result of adding the descriptions for syscalls of the mentioned syscall families. Few of them are yet to be fixed.
3045 &lt;ul&gt;
3046 &lt;li&gt;&lt;a href=&quot;https://syzkaller.appspot.com/bug?id=e70e6b077e9dac9807c86fd0aa418aa837180598&quot;&gt; compat_20_getfstat&lt;/a&gt;&lt;/li&gt;
3047 &lt;li&gt;&lt;a href=&quot;https://syzkaller.appspot.com/bug?id=ed011128934a1410452457bcebfe634fcf84b9a2&quot;&gt;clock_settime50&lt;/a&gt;&lt;/li&gt;
3048 &lt;li&gt;&lt;a href=&quot;https://syzkaller.appspot.com/bug?id=6d4983b443f3b9ec458313601e468b9cd4bf781e&quot;&gt;ioctl&lt;/a&gt;&lt;/li&gt;
3049 &lt;/ul&gt;
3050
3051 &lt;h2&gt;Stats&lt;/h2&gt;
3052 &lt;p&gt;Syscall coverage percent for NetBSD has now increased from nearly 30 to 44.96. Addition of compat syscalls resulted in getting a few new bugs.&lt;/p&gt;
3053 &lt;p&gt; Percentage of syscalls covered in few of the other Operating Systems:
3054 &lt;ul&gt;
3055 &lt;li&gt;Linux: 82&lt;/li&gt;
3056 &lt;li&gt;FreeBSD: 37&lt;/li&gt;
3057 &lt;li&gt;OpenBSD: 61&lt;/li&gt;
3058 &lt;/ul&gt;
3059 &lt;/p&gt;
3060 &lt;h2&gt;Conclusion&lt;/h2&gt;
3061 &lt;p&gt;In the next phase I would be working on generating the syscall descriptions using automation and adding ioctl device drivers with it&#8217;s help.&lt;/p&gt;
3062
3063 &lt;p&gt;Atlast, I would like to thank my mentors Cryo, Siddharth and Santhosh for their constant support and guidance.I am also thankful to NetBSD community for being kind to give me this opportunity of having an amazing summer this year.&lt;/p&gt;</content>
3064 </entry>
3065 <entry>
3066 <id>https://blog.netbsd.org/tnf/entry/gsoc_reports_extending_the_functionality</id>
3067 <title type="html">GSoC Reports: Extending the functionality of NetPGP, Part 1</title>
3068 <author><name>Kamil Rytarowski</name></author>
3069 <link rel="alternate" type="text/html" href="https://blog.netbsd.org/tnf/entry/gsoc_reports_extending_the_functionality"/>
3070 <published>2020-07-13T22:05:18+00:00</published>
3071 <updated>2020-07-13T22:20:52+00:00</updated>
3072 <category term="/The NetBSD Foundation" label="The NetBSD Foundation" />
3073 <category term="gsoc" scheme="http://roller.apache.org/ns/tags/" />
3074 <summary type="html">&lt;i&gt;This report was prepared by Jason High as a part of Google Summer of Code 2020&lt;/i&gt;
3075 &lt;p&gt;NetPGP is a library and suite of tools implementing OpenPGP under a BSD license. As part of Google Summer of Code 2020, we are working to extend its functionality and work towards greater parity with similar tools. During the first phase, we have made the following contributions&lt;/p&gt;
3076 &lt;ol&gt;
3077 &lt;li&gt;Added the Blowfish block cipher&lt;/li&gt;
3078 &lt;li&gt;ECDSA key creation&lt;/li&gt;
3079 &lt;li&gt;ECDSA signature and verification&lt;/li&gt;
3080 &lt;li&gt;Symmetric file encryption/decryption&lt;/li&gt;
3081 &lt;li&gt;S2K Iterated+Salt for symmetric encryption&lt;/li&gt;
3082 &lt;/ol&gt;</summary>
3083 <content type="html">&lt;i&gt;This report was prepared by Jason High as a part of Google Summer of Code 2020&lt;/i&gt;
3084 &lt;p&gt;NetPGP is a library and suite of tools implementing OpenPGP under a BSD license. As part of Google Summer of Code 2020, we are working to extend its functionality and work towards greater parity with similar tools. During the first phase, we have made the following contributions&lt;/p&gt;
3085 &lt;ol&gt;
3086 &lt;li&gt;Added the Blowfish block cipher&lt;/li&gt;
3087 &lt;li&gt;ECDSA key creation&lt;/li&gt;
3088 &lt;li&gt;ECDSA signature and verification&lt;/li&gt;
3089 &lt;li&gt;Symmetric file encryption/decryption&lt;/li&gt;
3090 &lt;li&gt;S2K Iterated+Salt for symmetric encryption&lt;/li&gt;
3091 &lt;/ol&gt;
3092 &lt;p&gt;ECDSA key generation is done using the &apos;--ecdsa&apos; flag with netpgpkeys or the &apos;ecdsa&apos; property if using libnetpgp&lt;/p&gt;
3093 &lt;pre&gt;&lt;code&gt;[jhigh@gsoc2020nb gsoc]$ netpgpkeys --generate-key --ecdsa --homedir=/tmp
3094 signature secp521r1/ECDSA a0cdb04e3e8c5e34 2020-06-25
3095 fingerprint d9e0 2ae5 1d2f a9ae eb96 ebd4 a0cd b04e 3e8c 5e34
3096 uid jhigh@localhost
3097 Enter passphrase for a0cdb04e3e8c5e34:
3098 Repeat passphrase for a0cdb04e3e8c5e34:
3099 generated keys in directory /tmp/a0cdb04e3e8c5e34
3100 [jhigh@gsoc2020nb gsoc]$
3101
3102 [jhigh@gsoc2020nb gsoc]$ ls -l /tmp/a0cdb04e3e8c5e34
3103 total 16
3104 -rw------- 1 jhigh wheel 331 Jun 25 16:03 pubring.gpg
3105 -rw------- 1 jhigh wheel 440 Jun 25 16:03 secring.gpg
3106 [jhigh@gsoc2020nb gsoc]$
3107 &lt;/code&gt;&lt;/pre&gt;
3108 &lt;p&gt;Signing with ECDSA does not require any changes&lt;/p&gt;
3109 &lt;pre&gt;&lt;code&gt;[jhigh@gsoc2020nb gsoc]$ netpgp --sign --homedir=/tmp/a0cdb04e3e8c5e34 --detach --armor testfile.txt
3110 signature secp521r1/ECDSA a0cdb04e3e8c5e34 2020-06-25
3111 fingerprint d9e0 2ae5 1d2f a9ae eb96 ebd4 a0cd b04e 3e8c 5e34
3112 uid jhigh@localhost
3113 netpgp passphrase:
3114 [jhigh@gsoc2020nb gsoc]$
3115
3116 [jhigh@gsoc2020nb gsoc]$ cat testfile.txt.asc
3117 -----BEGIN PGP MESSAGE-----
3118
3119 wqcEABMCABYFAl71EYwFAwAAAAAJEKDNsE4+jF40AAAVPgIJASyzuZgyS13FHHF/9qk6E3pYra2H
3120 tDdkqxYzNIqKnWHaB+a4J+/R7FkZItbC/EyXH5YA68AC1dJ7tRN/tJNIWfYjAgUb75SvM2mLHk13
3121 qmBo48S0Ai8C82G4nT7/16VF2OOUn7F/3XICghoQOyS1nxJilj8u2uphLOoy9VayL1ErORIZVw==
3122 =p30e
3123 -----END PGP MESSAGE-----
3124 [jhigh@gsoc2020nb gsoc]$
3125 &lt;/code&gt;&lt;/pre&gt;
3126 &lt;p&gt;Verification remains the same, as well.&lt;/p&gt;
3127 &lt;pre&gt;&lt;code&gt;[jhigh@gsoc2020nb gsoc]$ netpgp --homedir=/tmp/a0cdb04e3e8c5e34 --verify testfile.txt.asc
3128 netpgp: assuming signed data in &quot;testfile.txt&quot;
3129 Good signature for testfile.txt.asc made Thu Jun 25 16:05:16 2020
3130 using ECDSA key a0cdb04e3e8c5e34
3131 signature secp521r1/ECDSA a0cdb04e3e8c5e34 2020-06-25
3132 fingerprint d9e0 2ae5 1d2f a9ae eb96 ebd4 a0cd b04e 3e8c 5e34
3133 uid jhigh@localhost
3134 [jhigh@gsoc2020nb gsoc]$
3135 &lt;/code&gt;&lt;/pre&gt;
3136 &lt;p&gt;Symmetric encryption is now possible using the &apos;--symmetric&apos; flag with netpgp or the &apos;symmetric&apos; property in libnetpgp&lt;/p&gt;
3137 &lt;pre&gt;&lt;code&gt;[jhigh@gsoc2020nb gsoc]$ netpgp --encrypt --symmetric --armor testfile.txt
3138 Enter passphrase:
3139 Repeat passphrase:
3140 [jhigh@gsoc2020nb gsoc]$
3141
3142 [jhigh@gsoc2020nb gsoc]$ cat testfile.txt.asc
3143 -----BEGIN PGP MESSAGE-----
3144
3145 wwwEAwEIc39k1V6xVi3SPwEl2ww75Ufjhw7UA0gO/niahHWK50DFHSD1lB10nUyCTgRLe6iS9QZl
3146 av5Nji9BuQFcrqo03I1jG/L9s/4hww==
3147 =x41O
3148 -----END PGP MESSAGE-----
3149 [jhigh@gsoc2020nb gsoc]$
3150 &lt;/code&gt;&lt;/pre&gt;
3151 &lt;p&gt;Decryption of symmetric packets requires no changes&lt;/p&gt;
3152 &lt;pre&gt;&lt;code&gt;[jhigh@gsoc2020nb gsoc]$ netpgp --decrypt testfile.txt.asc
3153 netpgp passphrase:
3154 [jhigh@gsoc2020nb gsoc]$
3155 &lt;/code&gt;&lt;/pre&gt;
3156 &lt;p&gt;We added two new flags to support s2k mode 3: &apos;--s2k-mode&apos; and &apos;--s2k-count&apos;. See RFC4880 for details.&lt;/p&gt;
3157 &lt;pre&gt;&lt;code&gt;[jhigh@gsoc2020nb gsoc]$ netpgp --encrypt --symmetric --s2k-mode=3 --s2k-count=96 testfile.txt
3158 Enter passphrase:
3159 Repeat passphrase:
3160 [jhigh@gsoc2020nb gsoc]$
3161
3162
3163 [jhigh@gsoc2020nb gsoc]$ gpg -d testfile.txt.gpg
3164 gpg: CAST5 encrypted data
3165 gpg: encrypted with 1 passphrase
3166 this
3167 is
3168 a
3169 test
3170 [jhigh@gsoc2020nb gsoc]$
3171 &lt;/code&gt;&lt;/pre&gt;</content>
3172 </entry>
3173 <entry>
3174 <id>https://blog.netbsd.org/tnf/entry/gsoc_reports_curses_library_automated</id>
3175 <title type="html">GSoC Reports: Curses Library Automated Testing, Part 1</title>
3176 <author><name>Kamil Rytarowski</name></author>
3177 <link rel="alternate" type="text/html" href="https://blog.netbsd.org/tnf/entry/gsoc_reports_curses_library_automated"/>
3178 <published>2020-07-13T21:40:13+00:00</published>
3179 <updated>2020-07-13T22:21:34+00:00</updated>
3180 <category term="/The NetBSD Foundation" label="The NetBSD Foundation" />
3181 <category term="gsoc" scheme="http://roller.apache.org/ns/tags/" />
3182 <summary type="html">&lt;i&gt;This report was prepared by Naman Jain as a part of Google Summer of Code 2020&lt;/i&gt;
3183 &lt;h2&gt;&lt;a id=&quot;user-content-introduction&quot; class=&quot;anchor&quot; aria-hidden=&quot;true&quot; href=&quot;#introduction&quot;&gt;&lt;svg class=&quot;octicon octicon-link&quot; viewBox=&quot;0 0 16 16&quot; version=&quot;1.1&quot; width=&quot;16&quot; height=&quot;16&quot; aria-hidden=&quot;true&quot;&gt;&lt;path fill-rule=&quot;evenodd&quot; d=&quot;M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z&quot;&gt;&lt;/path&gt;&lt;/svg&gt;&lt;/a&gt;Introduction&lt;/h2&gt;
3184 &lt;p&gt;My GSoC project under NetBSD involves the development of test framework of curses library. Automated Test Framework (ATF) was introduced in 2007 but ATF cannot be used directly for curses testing for several reasons most important of them being curses has functions which do timed reads/writes which is hard to do with just piping characters to test applications. Also, stdin is not a tty device and behaves differently and may affect the results. A lot of work regarding this has been done and we have a separate test framework in place for testing curses.&lt;/p&gt;
3185 &lt;p&gt;The aim of project is to build a robust test suite for the library and complete the SUSv2 specification. This includes writing tests for the remaining functions and enhancing the existing ones. Meanwhile, the support for complex character function has to be completed along with fixing some bugs, adding features and improving the test framework.&lt;/p&gt;</summary>
3186 <content type="html">&lt;i&gt;This report was prepared by Naman Jain as a part of Google Summer of Code 2020&lt;/i&gt;
3187 &lt;h2&gt;&lt;a id=&quot;user-content-introduction&quot; class=&quot;anchor&quot; aria-hidden=&quot;true&quot; href=&quot;#introduction&quot;&gt;&lt;svg class=&quot;octicon octicon-link&quot; viewBox=&quot;0 0 16 16&quot; version=&quot;1.1&quot; width=&quot;16&quot; height=&quot;16&quot; aria-hidden=&quot;true&quot;&gt;&lt;path fill-rule=&quot;evenodd&quot; d=&quot;M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z&quot;&gt;&lt;/path&gt;&lt;/svg&gt;&lt;/a&gt;Introduction&lt;/h2&gt;
3188 &lt;p&gt;My GSoC project under NetBSD involves the development of test framework of curses library. Automated Test Framework (ATF) was introduced in 2007 but ATF cannot be used directly for curses testing for several reasons most important of them being curses has functions which do timed reads/writes which is hard to do with just piping characters to test applications. Also, stdin is not a tty device and behaves differently and may affect the results. A lot of work regarding this has been done and we have a separate test framework in place for testing curses.&lt;/p&gt;
3189 &lt;p&gt;The aim of project is to build a robust test suite for the library and complete the SUSv2 specification. This includes writing tests for the remaining functions and enhancing the existing ones. Meanwhile, the support for complex character function has to be completed along with fixing some bugs, adding features and improving the test framework.&lt;/p&gt;
3190 &lt;h2&gt;&lt;a id=&quot;user-content-why-did-i-chose-this-project&quot; class=&quot;anchor&quot; aria-hidden=&quot;true&quot; href=&quot;#why-did-i-chose-this-project&quot;&gt;&lt;svg class=&quot;octicon octicon-link&quot; viewBox=&quot;0 0 16 16&quot; version=&quot;1.1&quot; width=&quot;16&quot; height=&quot;16&quot; aria-hidden=&quot;true&quot;&gt;&lt;path fill-rule=&quot;evenodd&quot; d=&quot;M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z&quot;&gt;&lt;/path&gt;&lt;/svg&gt;&lt;/a&gt;Why did I chose this project?&lt;/h2&gt;
3191 &lt;p&gt;I am a final year undergraduate at Indian Institute of Technology, Kanpur. I have my majors in Computer Science and Engineering, and I am specifically interested in algorithms and computer systems. I had worked on building and testing a library on Distributed Tracing at an internship and understand the usefulness of having a test suite in place. Libcurses being very special in itself for the testing purpose interested me. Knowing some of the concepts of compiler design made my interest a bit more profound.&lt;/p&gt;
3192 &lt;h2&gt;&lt;a id=&quot;user-content-test-framwork&quot; class=&quot;anchor&quot; aria-hidden=&quot;true&quot; href=&quot;#test-framwork&quot;&gt;&lt;svg class=&quot;octicon octicon-link&quot; viewBox=&quot;0 0 16 16&quot; version=&quot;1.1&quot; width=&quot;16&quot; height=&quot;16&quot; aria-hidden=&quot;true&quot;&gt;&lt;path fill-rule=&quot;evenodd&quot; d=&quot;M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z&quot;&gt;&lt;/path&gt;&lt;/svg&gt;&lt;/a&gt;Test Framwork&lt;/h2&gt;
3193 &lt;p&gt;The testframework consists of 2 programs, director and slave. The framework provides its own simple language for writing tests. The slave is a curses application capable of running any curses function, while the director acts as a coordinator and interprets the test file and drives the slave program. The director can also capture slave&apos;s output which can be used for comparison with desired output.&lt;/p&gt;
3194 &lt;p&gt;&lt;a target=&quot;_blank&quot; rel=&quot;noopener noreferrer&quot; href=&quot;//netbsd.org/~kamil/gsoc_2020/director-slave.jpeg&quot;&gt;&lt;img src=&quot;//netbsd.org/~kamil/gsoc_2020/director-slave.jpeg&quot; alt=&quot;slave-director model&quot; title=&quot;Slave-Director model&quot; style=&quot;max-width:100%;&quot;&gt;&lt;/a&gt;&lt;/p&gt;
3195 &lt;p&gt;The director forks a process operating in pty and executes a slave program on that fork. The master side of pty is used for getting the data stream that the curses function call produces which can be futher used to check the correctness of behaviour. Director and slave communicate via pipes; command pipe and slave pipe. The command pipe carries the function name and arguments, while slave pipe carries return codes and values from function calls.&lt;/p&gt;
3196 &lt;p&gt;Let&apos;s walk through a sample test to understand how this works. Consider a sample program:&lt;/p&gt;
3197 &lt;pre&gt;&lt;code&gt;include start
3198 call win newwin 2 5 2 5
3199 check win NON_NULL
3200 call OK waddstr $win &quot;Hello World!&quot;
3201 call OK wrefresh $win
3202 compare waddstr_refresh.chk
3203 &lt;/code&gt;&lt;/pre&gt;
3204 &lt;p&gt;This is a basic program which initialises the screen, creates new window, checks if the window creation was successful, adds as string &quot;Hello World!&quot; on the window, refreshes the window, and compares it with desired output stored in check file. The details of the language can be accessed at &lt;a href=&quot;https://github.com/NetBSD/src/blob/trunk/tests/lib/libcurses/testframe.txt&quot;&gt;libcurses testframe&lt;/a&gt;.&lt;/p&gt;
3205 &lt;p&gt;The test file is interpreted by the language parser and the correponding actions are taken. Let&apos;s look how line #2 is processed. This command creates a window using &lt;code&gt;newwin()&lt;/code&gt;. The line is ultimately parsed as &lt;a href=&quot;https://github.com/NetBSD/src/blob/1d30657e76c9400c15eb4e4cfcdff2fea6c65a5a/tests/lib/libcurses/director/testlang_parse.y#L237&quot;&gt;&lt;code&gt;call: CALL result fn_name args eol&lt;/code&gt;&lt;/a&gt; grammar rule and executes the function &lt;a href=&quot;https://github.com/NetBSD/src/blob/1d30657e76c9400c15eb4e4cfcdff2fea6c65a5a/tests/lib/libcurses/director/testlang_parse.y#L1035&quot;&gt;&lt;code&gt;do_funtion_call()&lt;/code&gt;&lt;/a&gt;). Now, this function &lt;a href=&quot;https://github.com/NetBSD/src/blob/1d30657e76c9400c15eb4e4cfcdff2fea6c65a5a/tests/lib/libcurses/director/testlang_parse.y#L1048&quot;&gt;sends&lt;/a&gt; function name and arguments using command pipe to the slave. The slave, who is waiting to get command from the director, reads from the pipe and &lt;a href=&quot;https://github.com/NetBSD/src/blob/1d30657e76c9400c15eb4e4cfcdff2fea6c65a5a/tests/lib/libcurses/slave/slave.c#L144&quot;&gt;executes&lt;/a&gt; the command. This executes the correponding curses function from the &lt;a href=&quot;https://github.com/NetBSD/src/blob/1d30657e76c9400c15eb4e4cfcdff2fea6c65a5a/tests/lib/libcurses/slave/command_table.h#L40&quot;&gt;command table&lt;/a&gt; and the pointer to new window is &lt;a href=&quot;https://github.com/NetBSD/src/blob/1d30657e76c9400c15eb4e4cfcdff2fea6c65a5a/tests/lib/libcurses/slave/curses_commands.c#L3582&quot;&gt;returned&lt;/a&gt; via the slave pipe (&lt;a href=&quot;https://github.com/NetBSD/src/blob/1d30657e76c9400c15eb4e4cfcdff2fea6c65a5a/tests/lib/libcurses/slave/commands.c#L167&quot;&gt;here&lt;/a&gt;) after passing wrappers of functions. The director &lt;a href=&quot;https://github.com/NetBSD/src/blob/1d30657e76c9400c15eb4e4cfcdff2fea6c65a5a/tests/lib/libcurses/director/testlang_parse.y#L1140&quot;&gt;recieves&lt;/a&gt; them, and returned value is assigned to a variable(&lt;code&gt;win&lt;/code&gt; in line#2) or compared (&lt;code&gt;OK&lt;/code&gt; in line#4). This is the typical life cycle of a certain function call made in tests.&lt;/p&gt;
3206 &lt;p&gt;Along with these, the test framework provides capability to &lt;code&gt;include&lt;/code&gt; other test (line#1), &lt;code&gt;check&lt;/code&gt; the variable content (line#3), &lt;code&gt;compare&lt;/code&gt; the data stream due to function call in pty with desired stream (line#6). Tester can also provide inputs to functions via &lt;code&gt;input&lt;/code&gt; directive, perform delay via &lt;code&gt;delay&lt;/code&gt; directive, assign values to variables via &lt;code&gt;assign&lt;/code&gt; directive, and create a wide or complex charater via &lt;code&gt;wchar&lt;/code&gt; and &lt;code&gt;cchar&lt;/code&gt; directives respectively. The framework supports 3 kind of strings; null terminated string, byte string, and string of type chtype, based on the quotes enclosing it.&lt;/p&gt;
3207 &lt;h2&gt;&lt;a id=&quot;user-content-progress-till-the-first-evaluation&quot; class=&quot;anchor&quot; aria-hidden=&quot;true&quot; href=&quot;#progress-till-the-first-evaluation&quot;&gt;&lt;svg class=&quot;octicon octicon-link&quot; viewBox=&quot;0 0 16 16&quot; version=&quot;1.1&quot; width=&quot;16&quot; height=&quot;16&quot; aria-hidden=&quot;true&quot;&gt;&lt;path fill-rule=&quot;evenodd&quot; d=&quot;M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z&quot;&gt;&lt;/path&gt;&lt;/svg&gt;&lt;/a&gt;Progress till the first evaluation&lt;/h2&gt;
3208 &lt;h3&gt;&lt;a id=&quot;user-content-improvements-in-the-framework&quot; class=&quot;anchor&quot; aria-hidden=&quot;true&quot; href=&quot;#improvements-in-the-framework&quot;&gt;&lt;svg class=&quot;octicon octicon-link&quot; viewBox=&quot;0 0 16 16&quot; version=&quot;1.1&quot; width=&quot;16&quot; height=&quot;16&quot; aria-hidden=&quot;true&quot;&gt;&lt;path fill-rule=&quot;evenodd&quot; d=&quot;M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z&quot;&gt;&lt;/path&gt;&lt;/svg&gt;&lt;/a&gt;Improvements in the framework:&lt;/h3&gt;
3209 &lt;ol&gt;
3210 &lt;li&gt;Automated the checkfile generation that has to be done manually earlier.&lt;/li&gt;
3211 &lt;li&gt;Completed the support for complex chacter tests in director and slave.&lt;/li&gt;
3212 &lt;li&gt;Added features like variable-variable comparison.&lt;/li&gt;
3213 &lt;li&gt;Fixed non-critical bugs in the framework.&lt;/li&gt;
3214 &lt;li&gt;Refactored the code.&lt;/li&gt;
3215 &lt;/ol&gt;
3216 &lt;h3&gt;&lt;a id=&quot;user-content-testing-and-bug-reports&quot; class=&quot;anchor&quot; aria-hidden=&quot;true&quot; href=&quot;#testing-and-bug-reports&quot;&gt;&lt;svg class=&quot;octicon octicon-link&quot; viewBox=&quot;0 0 16 16&quot; version=&quot;1.1&quot; width=&quot;16&quot; height=&quot;16&quot; aria-hidden=&quot;true&quot;&gt;&lt;path fill-rule=&quot;evenodd&quot; d=&quot;M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z&quot;&gt;&lt;/path&gt;&lt;/svg&gt;&lt;/a&gt;Testing and bug reports&lt;/h3&gt;
3217 &lt;ol&gt;
3218 &lt;li&gt;Wrote tests for wide character routines.&lt;/li&gt;
3219 &lt;li&gt;Reported bugs (and possible fixes if I know):
3220 &lt;ul&gt;
3221 &lt;li&gt;&lt;a href=&quot;https://gnats.netbsd.org/cgi-bin/query-pr-single.pl?number=55433&quot; rel=&quot;nofollow&quot;&gt;lib/55433&lt;/a&gt; Bug in special character handling of ins_wstr() of libcurses&lt;/li&gt;
3222 &lt;li&gt;&lt;a href=&quot;https://gnats.netbsd.org/cgi-bin/query-pr-single.pl?number=55434&quot; rel=&quot;nofollow&quot;&gt;lib/55434&lt;/a&gt; Bug in hline() in libcurses [fixed]&lt;/li&gt;
3223 &lt;li&gt;&lt;a href=&quot;https://gnats.netbsd.org/cgi-bin/query-pr-single.pl?number=55443&quot; rel=&quot;nofollow&quot;&gt;lib/55443&lt;/a&gt; setcchar() incorrectly sets the number of elements in cchar [fixed]&lt;/li&gt;
3224 &lt;/ul&gt;
3225 &lt;/li&gt;
3226 &lt;/ol&gt;
3227 &lt;h2&gt;&lt;a id=&quot;user-content-project-proposal-and-references&quot; class=&quot;anchor&quot; aria-hidden=&quot;true&quot; href=&quot;#project-proposal-and-references&quot;&gt;&lt;svg class=&quot;octicon octicon-link&quot; viewBox=&quot;0 0 16 16&quot; version=&quot;1.1&quot; width=&quot;16&quot; height=&quot;16&quot; aria-hidden=&quot;true&quot;&gt;&lt;path fill-rule=&quot;evenodd&quot; d=&quot;M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z&quot;&gt;&lt;/path&gt;&lt;/svg&gt;&lt;/a&gt;Project Proposal and References:&lt;/h2&gt;
3228 &lt;ul&gt;
3229 &lt;li&gt;Proposal: &lt;a href=&quot;https://github.com/NamanJain8/curses/blob/master/reports/proposal.pdf&quot;&gt;https://github.com/NamanJain8/curses/blob/master/reports/proposal.pdf&lt;/a&gt;&lt;/li&gt;
3230 &lt;li&gt;Project Repo: &lt;a href=&quot;https://github.com/NamanJain8/curses&quot;&gt;https://github.com/NamanJain8/curses&lt;/a&gt;&lt;/li&gt;
3231 &lt;li&gt;Test Language Details: &lt;a href=&quot;https://github.com/NetBSD/src/blob/trunk/tests/lib/libcurses/testframe.txt&quot;&gt;https://github.com/NetBSD/src/blob/trunk/tests/lib/libcurses/testframe.txt&lt;/a&gt;&lt;/li&gt;
3232 &lt;li&gt;Wonderful report by Brett Lymn: &lt;a href=&quot;https://github.com/NamanJain8/curses/blob/master/reports/curses-testframe.pdf&quot;&gt;https://github.com/NamanJain8/curses/blob/master/reports/curses-testframe.pdf&lt;/a&gt;&lt;/li&gt;
3233 &lt;/ul&gt;</content>
3234 </entry>
3235 <entry>
3236 <id>https://blog.netbsd.org/tnf/entry/gsoc_reports_fuzzing_the_netbsd</id>
3237 <title type="html">GSoC Reports: Fuzzing the NetBSD Network Stack in a Rumpkernel Environment, Part 1</title>
3238 <author><name>Kamil Rytarowski</name></author>
3239 <link rel="alternate" type="text/html" href="https://blog.netbsd.org/tnf/entry/gsoc_reports_fuzzing_the_netbsd"/>
3240 <published>2020-07-13T14:58:58+00:00</published>
3241 <updated>2020-07-13T22:22:24+00:00</updated>
3242 <category term="/The NetBSD Foundation" label="The NetBSD Foundation" />
3243 <category term="gsoc" scheme="http://roller.apache.org/ns/tags/" />
3244 <summary type="html">&lt;i&gt;This report was prepared by Nisarg Joshi as a part of Google Summer of Code 2020&lt;/i&gt;
3245 &lt;p&gt;&lt;strong&gt;Introduction:&lt;/strong&gt;&lt;/p&gt;
3246 &lt;p&gt;The objective of this project is to fuzz the various protocols and layers of the network stack of NetBSD using rumpkernel. This project is being carried out as a part of GSoC 2020. This blog post is regarding the project, the concepts and tools involved, the objectives and the current progress and next steps.&lt;/p&gt;
3247 &lt;p&gt;&lt;strong&gt;Fuzzing:&lt;/strong&gt;&lt;/p&gt;
3248 &lt;p&gt;Fuzzing or fuzz testing is an automated software testing technique in which a program is tested by passing unusual, unexpected or semi-random input generated data to the input of the program and repeatedly doing so, trying to crash the program and detect potential bugs or undealt corner cases.&lt;/p&gt;
3249 &lt;p&gt;There are several tools available today that enable this which are known as fuzzers. An effective fuzzer generates semi-valid inputs that are &quot;valid enough&quot; in that they are not directly rejected by the parser, but do create unexpected behaviors deeper in the program and are &quot;invalid enough&quot; to expose corner cases that have not been properly dealt with.&amp;nbsp;&lt;/p&gt;
3250 &lt;p&gt;Fuzzers can be of various types like dumb vs smart, generation-based vs mutation-based and so on. A dumb fuzzer generates random input without looking at the input format or model but it can follow some sophisticated algorithms like in AFL, though considered a dumb fuzzer as it just flips bits and replaces bytes, still uses a genetic algorithm to create new test cases, where as a smart fuzzer will follow an input model to generate semi-random data that can penetrate well in the code and trigger more edge cases. Mutation and generation fuzzers handle test case generation differently. Mutation fuzzers mutate a supplied seed input object, while generation fuzzers generate new test cases from a supplied model.&lt;/p&gt;
3251 &lt;p&gt;Some examples of popular fuzzers are: AFL(American Fuzzy Lop), Syzkaller, Honggfuzz.&lt;/p&gt;</summary>
3252 <content type="html">&lt;i&gt;This report was prepared by Nisarg Joshi as a part of Google Summer of Code 2020&lt;/i&gt;
3253 &lt;p&gt;&lt;strong&gt;Introduction:&lt;/strong&gt;&lt;/p&gt;
3254 &lt;p&gt;The objective of this project is to fuzz the various protocols and layers of the network stack of NetBSD using rumpkernel. This project is being carried out as a part of GSoC 2020. This blog post is regarding the project, the concepts and tools involved, the objectives and the current progress and next steps.&lt;/p&gt;
3255 &lt;p&gt;&lt;strong&gt;Fuzzing:&lt;/strong&gt;&lt;/p&gt;
3256 &lt;p&gt;Fuzzing or fuzz testing is an automated software testing technique in which a program is tested by passing unusual, unexpected or semi-random input generated data to the input of the program and repeatedly doing so, trying to crash the program and detect potential bugs or undealt corner cases.&lt;/p&gt;
3257 &lt;p&gt;There are several tools available today that enable this which are known as fuzzers. An effective fuzzer generates semi-valid inputs that are &quot;valid enough&quot; in that they are not directly rejected by the parser, but do create unexpected behaviors deeper in the program and are &quot;invalid enough&quot; to expose corner cases that have not been properly dealt with.&amp;nbsp;&lt;/p&gt;
3258 &lt;p&gt;Fuzzers can be of various types like dumb vs smart, generation-based vs mutation-based and so on. A dumb fuzzer generates random input without looking at the input format or model but it can follow some sophisticated algorithms like in AFL, though considered a dumb fuzzer as it just flips bits and replaces bytes, still uses a genetic algorithm to create new test cases, where as a smart fuzzer will follow an input model to generate semi-random data that can penetrate well in the code and trigger more edge cases. Mutation and generation fuzzers handle test case generation differently. Mutation fuzzers mutate a supplied seed input object, while generation fuzzers generate new test cases from a supplied model.&lt;/p&gt;
3259 &lt;p&gt;Some examples of popular fuzzers are: AFL(American Fuzzy Lop), Syzkaller, Honggfuzz.&lt;/p&gt;
3260 &lt;p&gt;&lt;strong&gt;RumpKernel&lt;/strong&gt;&lt;/p&gt;
3261 &lt;p&gt;Kernels can have several different architectures like monolithic, microkernel, exokernel etc. An interesting architecture is the &amp;ldquo;anykernel&amp;rdquo; architecture, according to wikipedia, &amp;ldquo;The &quot;anykernel&quot; concept refers to an architecture-agnostic approach to drivers where drivers can either be compiled into the monolithic kernel or be run as a userspace process, microkernel-style, without code changes.&amp;rdquo; Rumpkernel is an implementation of this anykernel architecture. In case of the NetBSD rumpkernel, all the NetBSD subsystems like file system, network stack, drivers etc are compiled into standalone libraries that can be linked into any application process to utilize the functionalities of the kernel, like the file system or the drivers. This allows us to run and test various components of NetBSD kernel as a library linked to programs running in the user space.&lt;/p&gt;
3262 &lt;p&gt;This idea of rumpkernel is really helpful in fuzzing the components of the kernel. We can fuzz separate subsystems and allow it to crash without having to manage the crash of a running Operating System. Also the fact that context switching has an overhead in case syscalls are made to the kernel, Rumpkernel running in the userspace can eliminate this and save time.(Also since the spectre-meltdown vulnerabilities, system calls have become more costly due to the security reasons)&lt;/p&gt;
3263 &lt;p&gt;&lt;strong&gt;HonggFuzz + Rumpkernel Network Stack:&lt;/strong&gt;&lt;/p&gt;
3264 &lt;p&gt;In this project we will use the outlined Rumpkernel&amp;rsquo;s network stack and a fuzzer called the honggfuzz. Honggfuzz is a security oriented, feedback-driven, evolutionary, easy-to-use fuzzer. It is maintained by google.(&lt;a href=&quot;https://github.com/google/honggfuzz&quot;&gt;https://github.com/google/honggfuzz&lt;/a&gt;)&lt;/p&gt;
3265 &lt;p&gt;The project is hosted on github at: &lt;a href=&quot;https://github.com/NJnisarg/fuzznetrump/&quot;&gt;https://github.com/NJnisarg/fuzznetrump/&lt;/a&gt; .The Readme can help in setting it up. We first require NetBSD installed either on a physical machine or on a virtual machine like qemu. Then we will build the NetBSD distribution by grabbing the latest sources(&lt;a href=&quot;https://github.com/NetBSD/src&quot;&gt;https://github.com/NetBSD/src&lt;/a&gt;). We will enable fuzzer coverage by using MKSANITIZER and MKLIBCSANITIZER flags and using the ASan(Address) and UBSan(Undefined Behavior) sanitizers. These sanitizers will help the fuzzer in catching bugs related to undefined behavior and address and memory leaks. After that we will grab one of the fuzzer programs(example: src/hfuzz_ip_output_fuzz.c) and chroot into the newly built distribution from the host NetBSD OS. Then we will compile it using hfuzz-clang by linking the required rumpkernel libraries (for example in our case: rump, rumpnet, rumpnet_netinet and so on). This is where we use the rumpkernel as libraries linked to our program. The program will access the network stack of the linked rumpkernel and the fuzzer will fuzz those components of the kernel. The compiled binary will be run with honggfuzz. Detailed steps are outlined in the Readme of the linked repository.&lt;/p&gt;
3266 &lt;p&gt;&lt;strong&gt;Our Approach for network stack fuzzing:&lt;/strong&gt;&lt;/p&gt;
3267 &lt;p&gt;We have planned to fuzz various protocols at different layers of the TCP/IP stack. We have started with mostly widely used yet simple protocols like IP(v4), UDP etc. Along the progress of the project, we will be adding support for more L3(and above) protocols like ICMP, IP(v6), TCP as well as L2 protocols like Ethernet as a bit later phase.&lt;/p&gt;
3268 &lt;p&gt;The network stack has 2 paths:&amp;nbsp;&lt;/p&gt;
3269 &lt;ol&gt;
3270 &lt;li&gt;Input/ingress path&lt;/li&gt;
3271 &lt;li&gt;Output/egress path&lt;/li&gt;
3272 &lt;/ol&gt;
3273 &lt;p&gt;A packet is sent down the network stack via a socket from an application from the output path, whereas a packet is said to be received on a network interface into the network stack via the input path. Each network protocol has major input and output APIs for the packet processing. Example IP protocol has an ip_input() function to process an incoming packet and an ip_output() function to process an outgoing packet. We are planning to fuzz each protocol&amp;rsquo;s output and input APIs by sending the packet via the output path and receiving the packet via input path respectively.&amp;nbsp;&lt;/p&gt;
3274 &lt;p&gt;In order to fuzz the output and input path, the network stack setup and configuration we have is as follows:&lt;/p&gt;
3275 &lt;ul&gt;
3276 &lt;li&gt;We have a TUN device to which we can read and write a packet.&lt;/li&gt;
3277 &lt;li&gt;We have a socket that is bound to the TUN device, which can send and receive packets&lt;/li&gt;
3278 &lt;li&gt;In order to fuzz the input path, we &amp;ldquo;write&amp;rdquo; a packet to the TUN interface, which emulates a received packet on the network stack.&lt;/li&gt;
3279 &lt;li&gt;In order to fuzz the output path, we send a packet via the socket to the TUN interface to fuzz the output path.&lt;/li&gt;
3280 &lt;/ul&gt;
3281 &lt;p&gt;For carrying out all the above setup, we have separated out the common function for creating and configuring the TUN device and socket into a file called &amp;ldquo;net_config.c&amp;rdquo;&lt;/p&gt;
3282 &lt;p&gt;Also in order to reduce the rejection of packets carrying random data for trivial cases like checksum or header len, we have created functions that create/forge semi-random packets using the input data from honggfuzz. We manipulate certain fields in the packet to ensure that it does not get rejected trivially by the network stack and hence can reach and trigger deeper parts of the code. These functions for packet creations are located in the &amp;ldquo;pkt_create.c&amp;rdquo; file. For each protocol we fuzz, we add these functions to forge the headers and the packet. Currently we have support from UDP and IP(v4).&lt;/p&gt;
3283 &lt;p&gt;With these building blocks we have written programs like hfuzz_ip_output_fuzz.c, hfuzz_ip_input_fuzz.c etc, which setup the TUN device and socket using net_config.c and after taking the random data from honggfuzz, use it to forge a packet and send it down or up the stack. We compile these programs using hfuzz-clang as mentioned above and run it under honggfuzz to fuzz the network stack&amp;rsquo;s particular APIs.&lt;/p&gt;
3284 &lt;p&gt;&lt;strong&gt;Current Progress:&lt;/strong&gt;&lt;/p&gt;
3285 &lt;p&gt;Following things were worked upon in the first phase:&lt;/p&gt;
3286 &lt;ul&gt;
3287 &lt;li&gt;Getting honggfuzz functional for NetBSD(thanks to Kamil for those patches)&lt;/li&gt;
3288 &lt;li&gt;Coming up with the strategy for network configuration and packet creation. Writing utilities for the same.&lt;/li&gt;
3289 &lt;li&gt;Adding fuzzing code for protocols like IP(v4) and UDP.&lt;/li&gt;
3290 &lt;li&gt;Carrying out fuzzing for those protocols.&lt;/li&gt;
3291 &lt;/ul&gt;
3292 &lt;p&gt;&lt;strong&gt;Next Steps:&lt;/strong&gt;&lt;/p&gt;
3293 &lt;p&gt;As next steps following things are planned for upcoming phase:&lt;/p&gt;
3294 &lt;ul&gt;
3295 &lt;li&gt;Making changes and improvements by taking suggestions from the mentors.&lt;/li&gt;
3296 &lt;li&gt;Adding support for ICMP, IP(v6), TCP and later on for Ethernet.&lt;/li&gt;
3297 &lt;li&gt;Analyze and come up with effective ways to improve the fuzzing by focusing on the packet creation part.&lt;/li&gt;
3298 &lt;li&gt;Standardize the code to be extensible for adding future protocols.&lt;/li&gt;
3299 &lt;/ul&gt;</content>
3300 </entry>
3301 </feed>
3302
Response: application/gopher-menu
Original URLgopher://codemadness.org/1/git/sfeed_tests/file/input/sfe...
Content-Typeapplication/gopher-menu; charset=utf-8