code:
#+TITLE:Hurd Cheatsheat
#+AUTHOR:Joshua Branson
#+LATEX_HEADER: \usepackage{lmodern}
#+LATEX_HEADER: \usepackage[QX]{fontenc}
#+OPTIONS: H:10 toc:nil

* Introduction
The GNU Hurd is a modular based operating kernel built on top the micro-kernel GNU Mach.  The Hurd has numerous advantages over the traditional and monolithic design of a Unix kernel.  Various parts of the Hurd can be separated, loaded, unloaded, or replaced, in ways that monolithic kernels, like Linux cannot replicate.  If one part of the Hurd crashes, it won't crash the rest of the kernel.  It's also much smaller, elegant, and forward looking than Linux.

The Linux kernel is close to 15 or 16 million lines of code!  It is a massive black hole, that is only getting bigger and harder to maintain.  In contrast, the Hurd is smaller and is designed so that it can be written with in languages other than C. Applications can mount and start their own filesystem on the fly, and it is designed to lower the barrier of entry for young hackers kernel hackers.

Since Linux is monolithic, a small change can affect a very different part of the kernel in significant ways. This is why developers occasionally are discouraged from changing the existing code base.  On the Hurd, each server runs as a separate userspace process, which means new servers cannot easily affect, damage, or introduce bugs to the existing Hurd.  The Hurd even goes a step further and lets the user modify any existing server or use their own instead.

The Hurd is also more secure than a traditional Unix kernel.  A traditional unix user has certain permissions to read, write, or execute files they control.  Any time a user runs a program, traditionally a unix kernel runs that program with the full permissions of the original user.  This means any program that runs, can do things that the user could do!  Why should Firefox have the ability to access your gpg keys?  Why should gthumb be able to send email on your behalf?  Linux is actively trying to fix this problem, but many vulnerabilities still exist.  In contrast, the Hurd is designed to be a capability based system.  Applications run with the fewest permissions possible.

While there are numerous technological advantages to the Hurd, many of them have not been implemented yet.  The Hurd suffers from lack of funding and developers to extend it, but it is still an attractive platform to learn kernel hacking.  The Hurd allows one to modify running kernel servers, without affecting other users of the system without rebooting.  If a kernel server is running slowly or poorly, one can attaching GDB to the running process and debug it!  This is something that Linux cannot easily be modified to do.

# also use GNU coding standards
# https://www.gnu.org/prep/standards/html_node/

* Documentation things
:PROPERTIES:
:ID:       b28640cd-920f-4b3f-a84d-a4895bf91537
:END:
** dde is not really being maintained.  These are drivers from the linux kernel.  The Rump kernel seems to be a better option.
** Important programs to use
*** settrans
settrans [option] node [translator arg]
Settrans allows one to set a translator on a file or a directory.  If no translator is specified, then the existing translator on the node is cleared.

|---------+------------------------------------------------------------------|
| -a      | Start translator and set it as node's active translator          |
| -c      | create the node if it doesn't exist                              |
| -C      | chroot stuff.                                                    |
| -F      | starting a translator write its pid to a file                    |
| -L      | if a file already has a translator, put the new one on top of it |
| -o      | disconnect the old translator from the filesystem                |
| -p      | make the translator persistant across reboots                    |
| -P      | pause, don't start the translator until newline                  |
|         | this gives you time to start the debugger on a translator        |
| -s      | start the translator as active                                   |
| --stack | replace a translator                                             |
| -t      | timeout for translator startup time                              |
| -U      |                                                                  |
| -x      | only set the translator if there is not there one already        |
|         |                                                                  |
|         | If you are setting a passive translator, and node already has    |
|         | an active translator, then these options apply                   |
| -g      | go away                                                          |
| -k      | keep the translator running                                      |
|         |                                                                  |
|         | if there is an active translator                                 |
| -f      | force the translator to shutdown                                 |
| -R      | shutdown the translators beneath this one too                    |
| -S      | don't sync before killing                                        |
| -?      | give some help                                                   |
|---------+------------------------------------------------------------------|

By default a passive translator is set

*** showtrans
Shows the passive translators on a node.
*** fsysopts
Set the command line arguments for translators.  This lets you modify currently running translators on the fly!

I can use this to create a completed file for nginx.  I'll use either stowfs or unionfs to create a complete file.  Then when I add a new user, I'll use fsysopts to change the translator arguments.

fsysopts [option...] filesys [fs_option...]

This is actually quite cool!  Let's set up a file called hello.  Mount the hello translator on it, and then modify what the hello translator says!

$ touch hello
$ settrans hello /hurd/hello
$ showtrans hello
/hurd/hello
$ cat ./hello
"Hello, World!"
$ fsysopts ./hello
/hurd/hello --contents='Hello, World!
'

So now we know that we can modify the --contents value of the /hurd/hello translator.

$ fsysopts ./hello --contents="Hello, Joshua!
"
$cat ./hello
Hello Joshua!

** Vocab
*** Port things
**** Port
A port is a inter-process communication (IPC) channels implemented in the kernel.  They are similar to UNIX pipes.  Only one task can receive messages from a port.  Many tasks can send messages to the same port.
**** Port Bucket
A port bucket is a collection of ports with some extra data and an ability to lock the bucket so the ports cannot be used to communicate with any process.
**** Port Class
Is also a collection of ports, each of which can be conveniently manipulated.  A port class is not required to share any ports with a bucket and vice versa.
*** atomic operations vs. global locks

Global locks are one way to program via threads.  Some lock ensures that one thread has write access to some variable.  The other thread must wait until the lock is released.

Atomic operations is a way to write threads without using locks.  It is a way to guarantee that no write operation will cause a data race.  Essentially atomic operations ensure that a write operation occurs in one step relative to other threads.  It means that no thread can view the value while it is being changed.  I think it means that the write must be one line of assembly.

It sounds like a way or programming
*** File
Every file in the Hurd can be accessed via a port.  When a user wishes to access a file, it makes a remote procedure call to the server implementing the filesystem.

so file_t is actually mach_port_t.
*** Capabilities things
**** Capabilities
A capability is a process' right to perform various actions to another object.  A capability can be thought of as a car key.  The car key doesn't care who has the key.  If you have the key, then you can drive the car, listen to music, etc.

The Hurd has capabilities implemented as Mach ports.  BUT these ports are not persistent.  So when you reboot all those capabilities are lost.   To work around it, ACLs (Access Control Lists) are used to figure out who has permissions to do what.  For example my current home directory looks like

#+BEGIN_SRC sh :results output :dir ~/
ls -l
#+END_SRC

#+RESULTS:
#+begin_example
total 108
drwxr-xr-x  4 joshua home  4096 May 12 20:52 config-files
drwxr-xr-x  2 joshua home  4096 May 12 20:44 Desktop
drwxr-xr-x  3 joshua home  4096 Jun  8 14:55 documents
drwx------  2 joshua home  4096 Jun  5 22:18 Downloads
drwxr-xr-x 19 joshua home  4096 Jun  6 10:19 music
drwxr-xr-x  2 joshua home  4096 Jun  7 13:33 videos
#+end_example

Each node in the filesystem begins with lots of "r"s, "w"s, and "x"s.  These specify which users can read, write, or execute those nodes in the filesystem.

**** Capability operating system
Every process has some capabilities.  They have the least amount of capabilities in order to run.  These capabilities can be passed around so that other programs can have the same capabilities as other programs.

Modern OSs like UNIX, have the https://en.wikipedia.org/wiki/Confused_deputy_problem where one program tricks another.  This can only happen in an operating system that has Access control lists, like UNIX, but in a capability based system, this cannot happen.

Unfortunately the Hurd is not a true capability operating system.  To see how it might become a capability system, read the link.

http://lists.gnu.org/archive/html/l4-hurd/2005-10/msg00081.html

https://en.wikipedia.org/wiki/Object-capability_model
**** Principle of least privilege
Principle of least privilege, means that every program runs with the least amount of privileges it needs in order to run.

This is a security principle.  Firefox does not need to be able to access your gpg key, nor does your vlc need access to your webcam?
*** ABAC
Authorization-based access control

Access is granted by a server.  Some program wants access to some resource and an authorizing server create a capability that allows this to happen.  In the Hurd this is done with the auth server.
*** File Descriptor
When a process on UNIX attempts to open a file, the kernel accesses the file and returns a port.  The process does not directly have access to the hard drive.  So if you have 10 files open, then proc will have 10 entries like so: \\

=/proc/PID/fd/= \\

Allegedly, these are like capabilities, but they do not survive reboot.
*** Translator
A translator is a Hurd process that implements a filesystem interface.  When one accesses any file on disk, a translator provides the file.  Users can write their own translators that act on portions of the filesystem.  For example, one could write a translator that turns an xml act like a directory.  For example, suppose you have a xml file (file.xml) like so

#+BEGIN_HTML

 this is a file

 this is another file

#+END_HTML

If you make a translator responsible for serving it, then ~cd file.xml; ls~ would produce:

#+BEGIN_SRC latex
file1
file2
#+END_SRC
**** Active Translator
An active translator is one is currently operating on a filesystem node, but will be lost on reboot.
**** Passive Translator
A passive translator is one that is persistent after reboots.  If a passive translator is set for /home/joshua/documents that connects documents to an network file system, then whenever joshua tries to access his documents, if the translator is not already running, it will start to connect to the network file system to access joshua's documents.
**** Short Circuiting
https://www.gnu.org/software/hurd/hurd/translator/short-circuiting.html
The hurd has a symlink translator.  One can use it to create symlinks.  However, the Hurd will be faster if the filesystem handles this.  Instead of having extfs server start up a symlink server everytime it encounters a symlink, the ext2fs server can implement the task of the symlink server itself.  This means we do not have to start a symlink server and save some resources as well as minimize RPCs.

Other fs servers that might not support symlinks can still use the symlink translator.
**** Translator problems
**** TODO Translators set up by untrusted users
:LOGBOOK:
- State "TODO"       from              [2016-06-08 Wed 16:05]
:END:

I can write up the page to change the text on the wiki to be more readable.

Suppose you have a translator that implements a softlink to another node in the filesystem without revealing that that is what it is doing.  So /home/User2/documents/secretDir points to /home/User1.  If root does a rm -r /home/User2/, then /home/User1 gets all of his data deleted!

https://www.gnu.org/software/hurd/open_issues/translators_set_up_by_untrusted_users.html
**** Trust the behavior of Translators
https://www.gnu.org/software/hurd/open_issues/trust_the_behavior_of_translators.html
**** Translator pages I should read:
**** https://www.gnu.org/software/hurd/hurd/translator/writing/example.html
**** Setting up translators - HowTo
 https://www.gnu.org/software/hurd/hurd/translator/examples.html
Additional translators can be got from incubator, or hurd-extras.

cvs -z3 -d:pserver:anonymous@cvs.savannah.nongnu.org:/sources/hurdextras co <modulename>

    httpfs translator

$ settrans -a tmp/ /hurd/httpfs www.hurd-project.com/

or

$ settrans -a tmp/ /hurd/httpfs www.hurd-project.com/ --proxy=<proxy> --port=<port>
$ cd tmp/
$ ls -l

    ?ftpfs translator

$ settrans -cgap ftp /hurd/hostmux /hurd/ftpfs /
$ cd ftp
ftp$ ls
ftp$ cd ftp.fr.debian.org
ftp/ftp.fr.debian.org $ ls

    tarfs translator

You can use tarfs to mount (almost) any tar file (currently broken, 2010-08-25):

$ settrans -ca a /hurd/tarfs -z myfile.tar.gz
$ settrans -ca b /hurd/tarfs -y myfile.tar.bz2
$ settrans -ca c /hurd/tarfs myfile.tar

You can even use it to create new tar files:

$ settrans -ca new /hurd/tarfs -cz newfile.tar.gz
$ cp -r all my files  new/
$ syncfs new

This is not as fast as tar czvf newfile.tar.gz all my files, but at least it's more original. ;)

    cvsfs translator

$ settrans -ac cvsfs_testing /hurd/cvsfs cvs.savannah.nongnu.org /sources/hurdextras
$ cd cvsfs_testing

    pfinet translator -- configuring your network interface

$ settrans -fgca /servers/socket/2 /hurd/pfinet -i <interface> -a <ip address> -m <subnet mask> -g <gateway ip>

    Console translator -- setting up virtual consoles

$ console -d vga -d pc_mouse -d pc_kbd -d generic_speaker /dev/vcs

    ?iso9660fs translator -- 'mounting' your cdrom

$ settrans -ac /cdrom /hurd/iso9660fs /dev/<cdrom device file>

    ext2fs translator -- 'mounting' an ext2fs partition

$ settrans -ac /linux /hurd/ext2fs /dev/<partition device file>

    unionfs translator

To join "foo/" "bar/" and "baz/" in the directory "quux/", just do:

$ settrans -capfg quux/ /hurd/unionfs foo/ bar/ baz/

If you want to join even quux/ contents in the union itself, add -u as a translator argument. You can add filesystems at run-time with the fsysopts command.
**** TODO I can give more cool examples with translators
:LOGBOOK:
- State "TODO"       from              [2016-06-08 Wed 16:10]
:END:

https://www.gnu.org/software/hurd/hurd/documentation/translator_primer.html
**** https://www.gnu.org/software/hurd/hurd/translator.html
**** https://www.gnu.org/software/hurd/hurd/documentation/translators.html
**** https://www.gnu.org/software/hurd/hurd/debugging/translator.html
*** protid
A protid is short for a protected id.
*** Virtual File System
So no single process serves the filesystem.  Suppose a user wants to access the file /home/USERNAME/Documents/resume.pdf.  First the process looking for the file, sees if there is a translator mounted on /.  If there is not, then it tries to access /home.  If there is no translator there, it tries to access /home/USERNAME.  If there is no translator there, then it tries to access /home/USERNAME/Documents, etc.  The reason the Hurd works like this, is because a user can easily assign a translator at any given file or directory in the filesystem.

For example, consider that the user mounted an NSF filesystem (Network Filesystem) at /home/USERNAME/Documents.  When a process tries to access resume.pdf, then, if the nfs translator is started and mounts the remote filesystem.  When that happens the user can transparently open the remote file resume.pdf.
*** Computer Bought the Farm
Computer bought the farm means that a server crashed.  It is not a huge issue.  Just kill the translator and try again.

#+BEGIN_SRC sh :results output :exports both
settrans -fg FILE
#+END_SRC

*** Identity Based Access Control
The Identity Based Access control is when permissions are given to a process based on its identity.  This can be problematic if one program attempts to do something on behalf on another.  If Firefox wants to access my filesystem, but uses gpg to decrypt a file, gpg now is accessing a file.  BUT gpg doesn't have Firefox's ID.  So how does one get gpg to access the file but ONLY have the permissions that Firefox has?  This is called the Confused Deputy Problem.
*** Access Control Lists
An access control list says that a program can access a program if it has the proper permissions based on its ID.
*** real time things
**** real time operating system

This means that if an operating system to scheduled to do something at a specified time, then it must guarantee that the thing will be done very nearly at the time it said it would do it.  This means that the lag time must be very short.  Many modern operating systems need to be real time for gaming and video players.  Other wise the game/video can be choppy.

- react as quickly as possible to events
  This means that you might want to use a preemptible kernel.  AKA a kernel that has interupts.  AKA a kernel that can interuppt a running process, so that some other process can run.

  However, an interupt doesn't always mean that a process was preempted.  It's possible that an interupt will let the current process get back to the cpu.
- keep the highest priority activity running as much as possible
**** realtime

An OS does things when it is scheduled to do them.

You need some global locks for this to work...
*** locks
**** mcs locks
Linux uses mcs locks
**** ticket locks
have annoying limitations
*** page
*** stack
*** heap
** Hurd
*** Things the Hurd can do
**** SATA drives
A SATA drive driver was added in 10th May 2013.
It should work for many SATA drives.
**** The Hurd now supports 128GB partition sizes
**** types of virtualization
***** subhurds
:PROPERTIES:
:ID:       0cca2f9b-b92b-4508-828f-3f6f19883a79
:END:
https://www.gnu.org/software/hurd/hurd/subhurd.html

Run two instances of the hurd ontop of the same mach.  This is great for debugging purposes and probably other things.

I need to use qemu to provide a new partition that uses the same qemu image or a different one.  If a use the same qemu image, then the subhurd won't be able to write any changes.  So it's just better to have two different images.  One for the main hurd and another for the subhurd.

I'll need to tell qemu to add the image.

#+BEGIN_SRC sh :results output :exports both
-hdd FILE
#+END_SRC

Then I'll need to find the root filesystem of that extra debian image.

#+BEGIN_SRC sh :results output :exports both /ssh:root@localhost#2222:/home/joshua
fdisk -l /dev/hd1
#+END_SRC

#+RESULTS:

Then I'll just set the ext2fs server running like so:

#+BEGIN_SRC sh :results output :exports both
su settrans -c /home/joshua/hd1s1 /hurd/ext2fs /dev/hd1s1
#+END_SRC

Then in the hurd I can just run boot /dev/hd0s6  or whatever the file name is.

#+BEGIN_SRC sh :results output :exports both
boot hd1s1
#+END_SRC

BUT the terminal that runs that command, will take all new commands to run in the subhurd.
So I'll need to ssh into the hurd with another terminal, then run boot.  BUT that might not really work either.  Because
by default the subhurd does not have internet access...

To exit or close the subhurd, I'll have to run halt or reboot.
***** remap one service for another
*** Hurd Servers
Some Hurd servers can be found in $(HURD)/trans/

**** To get help about how to use a server run

#+BEGIN_SRC sh :results output :exports both :dir /ssh:joshua@localhost#2222:/home/joshua
/hurd/hello --help
#+END_SRC

#+RESULTS:
#+begin_example
Usage: hello [OPTION...]
A translator providing a warm greeting.

  -c, --contents=STRING      Specify the contents of the virtual file
  -?, --help                 Give this help list
      --usage                Give a short usage message
  -V, --version              Print program version

Mandatory or optional arguments to long options are also mandatory or optional
for any corresponding short options.

Report bugs to <bug-hurd@gnu.org>.
#+end_example

**** Auth
This is apparently a Access Control Lists server.  Suppose Firefox wishes to access a file on disk.  Firefox requests the file from the filesystem.  The filesystem wants to be sure that Firefox can do this, so it asks the Auth server.
https://www.gnu.org/software/hurd/hurd/documentation/auth.html

3 things the auth server can do:

- If it receives two authentication ports, then the server will provide a 3rd ports that is the union of the first sets of uids and guids.
- Any user with uid 0 can create a arbitrary authentication port.
- Allows any two arbitrary programs to trust one another.
**** remap
This is a way of substituting one service for another

hurd-2017-03-17-13:28:17< teythoon> do you know how to use 'remap' to test your translator ?
hurd-2017-03-17-13:28:42< teythoon> eth-multiplexer will help too
hurd-2017-03-17:13:29:45< teythoon> see http://darnassus.sceen.net/~hurd-web/hurd/subhurd/#index4h2 for how to setup eth-multiplexer on your hurd machine
hurd-2017-03-17-13:31:59< teythoon> then you can settrans -ac my-2 path/to/my/ip-stack -what -ever ; remap /servers/socket/2 $(pwd)/my-2 -- ping 8.8.8.8
hurd-2017-03-17:13:39:18< jlledom> I didn't know the eth-multiplexer, I configured two interfaces in my virtual machine and remap exactly that way

**** process server
This is an information server.

- It keeps track of host information: hostname, hostid, and system version.
- It takes care of POSIX sessions and session groups.  Some programs will want this information.
- The process server maintains a 1-1 mapping between mach tasks and hurd processes.  Each process can register a port with the process server, which allows anyone to communicate to that process.  That means each process must implement security stuff themselves.
- It also does process collections?  http://darnassus.sceen.net/~hurd-web/hurd-paper/

The process server is optional.  Mach has some kind of restrictions that means a user must be root to see all the processes.

BUT now I'm not certain why that is a problem...I feel like most users wouldn't need to know all the running processes...
**** Users can have more than one uid
**** password
found in /servers/password

This checks to see if a user's password is the correct password.

It also checks to see if a group's password is the real password.
**** pfinet
This is the TCP/IP networking stack.  It is using dde.
**** symlink

Set up a symlink.  Apparently it's much faster if the filesystem does this, but you can let the symlink translator create a symlink for you.  This can be useful, if you are developing a filesystem, but don't want to create a symlink yet.
#+BEGIN_SRC sh :results output :exports both
settrans -a link-to-programming /hurd/symlink /home/joshua/programming/
#+END_SRC
**** the hurd servers parse arguments with argp.  I can learn more about it here:

[[info:libc#Argp][info:libc#Argp]]
**** crash server

You can use the crash server very cool!

https://savannah.gnu.org/bugs/?17319

#+BEGIN_SRC sh :results output :exports both :dir ~/programming/c/
cat crash.c
#+END_SRC

#+RESULTS:
#+begin_example
// this doesn't seem to actually crash c...
/* int main (void) */
/* { */
/*   return (int ) 0; */

/* } */
// this will actually crash a c program
int main ()
{
  return 1/0;
}
#+end_example

Running the program, will create a file called core.

ls -l core

ls -l /servers/crash

*** Hurd translators
**** hello
#+BEGIN_SRC sh :results output :exports both
touch hello
cat hello
settrans -a ./hello /hurd/hello
cat hello # Hello World!
settrans -g ./hello
cat hello
#+END_SRC
**** xmlfs
The xmlfs documentation can be found here:  http://cvs.savannah.gnu.org/viewvc/*checkout*/hurdextras/xmlfs/README
The example xml file can be found here:   http://cvs.savannah.gnu.org/viewvc/*checkout*/hurdextras/xmlfs/example.xml

xmlfs allows one to create a directory tree out of an xml file.  Cool eh?

	xmlfs - a translator for accessing XML documents

  Copyright (C) 2002, 2005 HurdFR
  Written by Manuel Menal          <mmenal@hurdfr.org>
         and Marc de Saint Sauveur <marc@hurdfr.org>

This is  only an alpha version.   It works in read  only.  It supports
text  nodes and  attributes. It  doesn't do  anything fancy  like size
computing, though. Here is an example of how to use it:

$ settrans -ca xml /hurd/xmlfs example.xml  #the website says to use ./xmlfs
$ cd xml; ls
library0 library1
$ cd library0; ls -A
.text1  .text2  @name  book0  book1  book2  sub-library0  sub-library1
$ cat .text2

CDATA, again !

$ cat book0
<book>
<author>Mark Twain</author>
<title>La case de l'oncle Tom</title>
<isbn>4242</isbn>
</book>
$ cat book0/author/.text
Mark Twain

As  you can  see,  text nodes  are  named .textN,  with  N an  integer
starting from 0. Sorting is supposed to be stable, so you get the same
N every time you access the same  file. If there is only one text node
at this level, N is ommitted. Attributes are prefixed with @.

An  example file,  example.xml, is  provided. Of  course, it  does not
contain anything  useful. xmlfs  has been tested  on several-megabytes
XML documents, though.

Comments are welcome.

	-- Manuel Menal <mmenal@hurdfr.org>
**** stowfs
One can create an arbitrary directory tree on the fly.
**** run
This just outputs the result of running a program.
**** unionfs
*** Hurd Libraries

**** Trivfs
This is the library that is used to create simple translators that operate on single filesystem nodes.

It has some weird callback function names though:

- =trivfs_S_io_read=
  | struct trivfs_protid   | the credentials you have                |
  | mach_port_t            | the port where the reply will be sent.  |
  | mach_msg_type_name_t   | the rights we have on the above port    |
  | vm_address *           | pointer to where we write the data      |
  | mach_msg_type_number_t | here you store how much data you return |
  | off_t                  | Seek a position.                        |
  | mach_msg_type_number_t | how much data you should write.         |
the io_read is the name of the RPC.  The S prefix means this is the _S_erver stub here.  the trivfs__ means that this is not the bare RPC, but that mach_port_t is converted to a credential.
***** Real basic trivfs stuff
A trivfs translator needs to define certain functions and variable names, then call argp_parse (see argp in the gnu c library) to parse command line arguments.

redefine extern struct arpg * trivfs_runtime_argp to add option parsing to a trivfs program.

After I run argp, I should check to see if my translator was run as a program or started via settrans.

trivfs_startup tries to start the translator.
***** various functions and variables that must be defined
- extern int trivfs_fstype  usually FSTYPE_MISC
  other options found in $(HURD)/hurd/hurd_types.h
  There's one for tar, ext2fs, and one for a socket.  cool!

- extern int trivfs_fsid
- extern int trivfs_allow_open
  Specify if a translator's node can be read, written to, or executed. Set this to a bitwise OR combination of O_READ, O_WRITE, O_EXEC.  Trivfs will only allow opening the file this way.

The following must also be defined, but they say which of the three rwx, are implemented in code.
- extern int trivfs_support_read
- extern int trivfs_support_write
- extern int trivfs_support_exec

- void trivfs_modify_stat
  This should modify the filesystem node for use by io_stat.  This can be a blank function, but it needs to be defined.
- error_t trivfs_goaway
  The function that is called when the translator should die.  It can just return 0.
***** Trivfs has some options.   Read about it on p. 35 of the hurd reference manual.
**** libnetfs
A library used for implementing network filesystems, where the translator does not directly control the data.

This is how ftpfs, nfs, and unionfs are written.
**** libdiskfs
a library used for making filesystems where the translator does control the underlying data.
**** libports
Functions for working with ports.  It's an abstraction of what the hurd expects from a message passing system.
**** libstore
A low level interface to a fixed block or storage, which can be read or written to.
**** libiohelp
A library for helping servers with io.
**** libtreefs  is this still defunct?
**** libthreads
The c thread library
**** libpthread
the p thread library
**** libhurduser
**** fshelp
A library to help with the file protocol.
**** libihash
integer-keyed hash table
**** libps
library to print process information
**** libshouldbeinlibc
A library of functions that should be in libc
**** libpipe
*** libpcap doesn't work on the hurd?  What's that?
*** The Hurd Interfaces
The Hurd provides interfaces so that one can create remote procedure calls.  Most libc functions are translated into GNU/Hurd RPCs.
*** Hurd functions
**** io_stat
is an RPC
**** file_chown
**** file_chauthor
**** file_chmod
*** Writing C programs in the Hurd
If you want your program to run on the Hurd, then the top of your file needs to be:

#+BEGIN_SRC C
#define _GNU_SOURCE 1
#+END_SRC

You can compile a program with ~gcc -g -o <program name> <program file>~.  For example, ~gcc -g -o cat cat.c~.

Alternatively, you can compile programs via

=gcc -g -D_GNU_SOURCE -o <program name> <program file>=

**** Read the hurd *defs. These are functions that you can call.
#+BEGIN_SRC sh :results output :exports both :dir /ssh:joshua@localhost#2222:/home/joshua/programming/hurd/hurd
ls *defs
#+END_SRC

#+RESULTS:
#+begin_example
auth.defs		  hurd_types.defs   msg_request.defs
auth_reply.defs		  ifsock.defs	    newterm.defs
auth_request.defs	  iioctl.defs	    password.defs
crash.defs		  interrupt.defs    pfinet.defs
crash_reply.defs	  io.defs	    process.defs
default_pager.defs	  io_reply.defs     process_reply.defs
default_pager_reply.defs  io_request.defs   process_request.defs
exec.defs		  ioctls.defs	    socket.defs
exec_startup.defs	  kdioctl.defs	    startup.defs
fs.defs			  kernel_boot.defs  startup_notify.defs
fs_notify.defs		  login.defs	    startup_reply.defs
fsys.defs		  msg.defs	    term.defs
fsys_reply.defs		  msg_reply.defs    tioctl.defs
#+end_example

**** read the navigating document
[[file:/ssh:joshua@localhost#2222:/home/joshua/programming/hurd/doc/navigating::Some%20ideas%20on%20navigating%20and%20understanding%20the%20Hurd%20source%20code.][here.]]

#+BEGIN_SRC sh :results output :exports both :dir /ssh:joshua@localhost#2222:/home/joshua/programming/hurd/doc/
cat navigating
#+END_SRC

#+RESULTS:
#+begin_example
Some ideas on navigating and understanding the Hurd source code.

First you must understand Mach IPC and MiG.  Pay special attention to
the way that various kinds of MiG calls manage memory; this is crucial
to avoid leaks.  The "Mach server writers' guide" explains all this.

Most programs that make up the Hurd are built out of libraries; a
solid understanding of the libraries that make up the source is
essential.  First start by reading the libports library specification
(in libports/ports.h).  This library manages the Mach ports that
servers handle.

Then start looking at some Hurd interfaces.  A good place to start is
to look at the proc server.  There is only one proc server in a
running system, but examine the way the interface (hurd/process.defs)
is written and the way the proc server implements it.

Then look at the auth server; make sure you understand how an auth
transaction works.  Again, by looking at the implementation, you can
see a simple example.

Filesystems are more complex; the interface specification is in
hurd/io.defs and hurd/fs.defs.  These interfaces are implemented by
three different libraries: trivfs, diskfs, and netfs.  trivfs
implements single-node filesystems (that thus have no directories).
Most trivfs filesystems don't even do any filesystem stuff at all.
See, for example, the null translator (trans/null.c) for a simple
example of using trivfs.

diskfs is used for disk-based filesystems, with one in existence now:
ext2fs.  If you write another diskfs-based filesystem, you
should DEFINITELY imitate the algorithms found in ext2fs; this
is crucial to getting locking right.

netfs is used for nfs and other such things: with directories, and all
the actual filesystem operations being done by some other program (not
necessarily over a network).  The nfs implementation is fairly easy to
understand.

Examine some translators in the trans directory to see various simple
examples.

Also very important is to acquire familiarity with the Hurd and Mach
calls provided in the GNU C library.  Look at the header files in libc
to see what they are, and read through them.  Also examine parts of
the libc implementation whenever you have any doubt about what an
interface call should do: find where the C library uses that call, and
that should help.  It's worth, in fact, spending some time just
exploring the implementation of various things in the hurd C library.

You should take a look at all the libraries in the Hurd; spend time
reading the code.  Feel free to ask questions to help understand what
you read.
#+end_example

**** filesystem
***** Get the size of a file
#+BEGIN_SRC C :dir /ssh:joshua@localhost#2222:/home/joshua/programming/hurd/hurd
#define _GNU_SOURCE 1

mach_msg_type_number_t amount;
error_t err;

err = io_readable (f, &amount);
#+END_SRC

#+RESULTS:

***** read a file
#+BEGIN_SRC C :dir /ssh:joshua@localhost#2222:/home/joshua/programming/hurd/hurd
#define _GNU_SOURCE 1

mach_msg_type_number_t amount;
error_t err;

err = io_read (f, &buf, &amount, -1, amount);
#+END_SRC
*** how a program is run in the hurd
A program is run via the file_exec RPC.  The fileserver verifies that the user can run the program (via auth probably), then modify some ports for setuid execution, then invoke the standard execserver found in /servers/exec.
*** rpctrace
compile a translator

gcc -g -Wall -D_GNU_SOURCE -o looper looper.c
touch node

Set a translator.
settrans -a node /bin/rpctrace ./looper /

Then running the translator should trigger rpctrace.
*** simple malicious translator that could break stuff:

https://www.sceen.net/the-trust-tree/

A user could create a translator at a file in /tmp.  A program meant to clean /tmp, would try to remove the file, but the translator would just not properly return from a trivfs function it had recorded.

This could cause the /tmp cleaner to crash, which could potentially stop other programs from using /tmp.
*** youpi ported the hurd to xen for driver stuff
hurd-2017-02-28-15:47:30< teythoon> the reason why youpi has focussed on xen is driver support
hurd-2017-02-28-15:47:31< ram_> I'm working on compiling from source right now, and getting a few simple patches in.
hurd-2017-02-28-15:48:10< ram_> How does xen driver support help the grand scheme of things?
hurd-2017-02-28:15:48:27< teythoon> we have a bunch of Linux2.0 drivers in gnumach that will not work on amd64
hurd-2017-02-28-15:48:55< teythoon> with the xen port we don't need them, as we use xens faacilities
hurd-2017-02-28-15:50:13< ram_> I'll have to read up on that. So what about Intel x86-64 support? That's not something you guys want?
hurd-2017-02-28-15:50:15< youpi> well, it's not really the reason why I started with the xen 64bit port
hurd-2017-02-28-15:50:31< youpi> it's just that it skips a lot of 64bit initialization stuff
hurd-2017-02-28-15:50:44< youpi> the very task of booting from grub to a 64bit kernel is not trivial :)
hurd-2017-02-28-15:50:52< youpi> so it was easier to just boot from xen
*** teythoon  hurd test? link?
hurd-2017-02-28-17:50:13< teythoon> fwiw, i have https://bitbucket.org/teythoon/hurdtest that i still use

*** grep-ing to the irc stuff:

look for something     specify year
=grep -C 10 'x15*' * | grep '2015' | grep -C 10 'x15' | grep -v "has joined" | grep -v "has quit" | less=
*** the hurd uses multiplexs over lots of threads...not the way mach was intended to be used:

hurd-2016-12-10-13:59:28< braunr> not enough to really know about the project though
hurd-2016-12-10-13:59:34< braunr> so i couldn't answer teythoon
hurd-2016-12-10-14:21:17< teythoon> i was a bit jealous... must have been nice :)
hurd-2016-12-10-14:22:24< braunr> not really
hurd-2016-12-10-14:22:48< braunr> it would have been if i could get what was going on :)
hurd-2016-12-10-14:24:19< teythoon> i mean meeting to work on something together is nice
hurd-2016-12-10-14:24:33< teythoon> which reminds me, who is going to fosdem ?
hurd-2016-12-10-14:25:00< braunr> not this year
hurd-2016-12-10-14:25:02< braunr> probably the next
hurd-2016-12-15-17:37:04< braunr> i'm pretty sure that i read something saying the original devs wanted sync RPCs, and didn't care much about whether IPCs were sync or not
hurd-2016-12-15-17:37:34< braunr> hat's also the recommendation from mach docs iirc
hurd-2016-12-15-17:37:34< braunr> that*
hurd-2016-12-15-17:37:45< braunr> which is why most RPCs are declared as routines
hurd-2016-12-15-17:39:12< antrik> it combines the complexity of async IPC with the complexity of threading
hurd-2016-12-15-17:41:26< braunr> i don't think it does
hurd-2016-12-15-17:41:45< braunr> from the point of view of the hurd developer writing his server, IPC can be considered sycnrhonous
hurd-2016-12-15-17:42:22< braunr> debugging is sometimes affected
hurd-2016-12-15-17:43:54< antrik> well, the problem is exacerbated by the fact that the Hurd doesn't use a thread per server-side object (which seems to be the idiom intended by the Mach developers), but rather multiplexes incoming requests over a pool of threads not bound to particular objects
hurd-2016-12-15-17:46:07< braunr> that, i agree
hurd-2016-12-15:17:46:28< braunr> in x15, i intend to make server thread bind to client threads
hurd-2016-12-15-17:46:37< braunr> through something i would call "thread binding"
hurd-2016-12-15-17:46:51< antrik> if you have a thread per object, it probably makes sense to pretend RPCs are sync (though in this case it could be argued that they should rather be in separate processes, like in EROS...)
hurd-2016-12-15-17:47:12< braunr> yes, a process per client/object also fits nicely
hurd-2016-12-15-17:47:20< braunr> especially when we can move receive rights
hurd-2016-12-15-17:47:36< antrik> however, in a multiplexed model like in the Hurd, using threading instead of an event-drived model makes no sense
hurd-2016-12-15-17:48:22< antrik> (of course, the fact that FS servers rely heavily on transparent I/O through memory mapping makes this tricky...)
hurd-2016-12-15-17:48:32< braunr> no i still disagree with that
hurd-2016-12-15-17:48:37< braunr> threading makes sense
hurd-2016-12-15-17:48:44< braunr> mapping threads to messages doesn't
hurd-2016-12-15-17:49:18< antrik> I'm talking about having a thread per request
hurd-2016-12-10-13:59:28< braunr> not enough to really know about the project though
hurd-2016-12-10-13:59:34< braunr> so i couldn't answer teythoon
hurd-2016-12-10-14:21:17< teythoon> i was a bit jealous... must have been nice :)
hurd-2016-12-10-14:22:24< braunr> not really
hurd-2016-12-10-14:22:48< braunr> it would have been if i could get what was going on :)
hurd-2016-12-10-14:24:19< teythoon> i mean meeting to work on something together is nice
hurd-2016-12-10-14:24:33< teythoon> which reminds me, who is going to fosdem ?
hurd-2016-12-10-14:25:00< braunr> not this year
hurd-2016-12-10-14:25:02< braunr> probably the next
hurd-2016-12-15-17:37:04< braunr> i'm pretty sure that i read something saying the original devs wanted sync RPCs, and didn't care much about whether IPCs were sync or not
hurd-2016-12-15-17:37:34< braunr> hat's also the recommendation from mach docs iirc
hurd-2016-12-15-17:37:34< braunr> that*
hurd-2016-12-15-17:37:45< braunr> which is why most RPCs are declared as routines
hurd-2016-12-15-17:39:12< antrik> it combines the complexity of async IPC with the complexity of threading
hurd-2016-12-15-17:41:26< braunr> i don't think it does
hurd-2016-12-15-17:41:45< braunr> from the point of view of the hurd developer writing his server, IPC can be considered sycnrhonous
hurd-2016-12-15-17:42:22< braunr> debugging is sometimes affected
hurd-2016-12-15-17:43:54< antrik> well, the problem is exacerbated by the fact that the Hurd doesn't use a thread per server-side object (which seems to be the idiom intended by the Mach developers), but rather multiplexes incoming requests over a pool of threads not bound to particular objects
hurd-2016-12-15-17:46:07< braunr> that, i agree
hurd-2016-12-15:17:46:28< braunr> in x15, i intend to make server thread bind to client threads
hurd-2016-12-15-17:46:37< braunr> through something i would call "thread binding"
hurd-2016-12-15-17:46:51< antrik> if you have a thread per object, it probably makes sense to pretend RPCs are sync (though in this case it could be argued that they should rather be in separate processes, like in EROS...)
hurd-2016-12-15-17:47:12< braunr> yes, a process per client/object also fits nicely
hurd-2016-12-15-17:47:20< braunr> especially when we can move receive rights
hurd-2016-12-15-17:47:36< antrik> however, in a multiplexed model like in the Hurd, using threading instead of an event-drived model makes no sense
hurd-2016-12-15-17:48:22< antrik> (of course, the fact that FS servers rely heavily on transparent I/O through memory mapping makes this tricky...)
hurd-2016-12-15-17:48:32< braunr> no i still disagree with that
hurd-2016-12-15-17:48:37< braunr> threading makes sense
hurd-2016-12-15-17:48:44< braunr> mapping threads to messages doesn't
hurd-2016-12-15-17:49:18< antrik> I'm talking about having a thread per request
*** make the Hurd fsck etx2 by default
hurd-2017-01-10-13:13:38< braunr> we tend to set FSCKFIX=yes in /etC/default/rcS
*** Debugging
**** debugging Translators

In order to debug translators and being able to step into glibc during it, on Debian you need the hurd-dbg and libc0.3-dbg packages installed. If you need to debug the initialization of the translator, start the translator like

$ settrans -Pa /foo /usr/bin/env LD_LIBRARY_PATH=/usr/lib/debug /hurd/foofs

The -P option will make it pause and you will be able to attach ?GDB to the process.

For example

settrans -g -o /hurd/one one.c

gdb one 357

https://www.gnu.org/software/hurd/hurd/debugging/translator.html
**** Debugging the hurd with a subhurd
https://www.gnu.org/software/hurd/hurd/debugging/subhurd.html
https://www.gnu.org/software/hurd/hurd/subhurd.html

[[subhurds]]
**** how to provide a backtrace
https://www.gnu.org/software/hurd/gdb/backtrace.html
**** debugging glibc changes
https://www.gnu.org/software/hurd/hurd/debugging/glibc.html
**** hardward watchpoint
https://www.gnu.org/software/hurd/hurd/debugging/hardware_watchpoint.html
**** you can also use crash server
*** what is gsync?
*** building the Hurd try: apt-get build-dep hurd
hurd-2017-02-28:21:25:56< teythoon> and the same for gnumach if you want to build that
hurd-2017-02-28-21:26:01< teythoon> then clone the repository
hurd-2017-02-28-21:26:08< teythoon> autoreconf -fi
hurd-2017-02-28-21:26:22< teythoon> the usual ./configure, make dance
hurd-2017-02-28-21:26:46< teythoon> building it shouldn't be too problematic
hurd-2017-02-28-21:27:02< teythoon> building something that boots is a bit more tricky
hurd-2017-02-28-21:27:08< teythoon> good luck ;)
*** testing lwip
hurd-2017-06-03:15:26:28< teythoon> teythoon@hurdbox ~/build/hurd-upstream % remap /servers/socket/2 /tmp/t -- /usr/bin/curl http://192.168.122.1:8123/t
*** git send email not supported on the Hurd
hurd-2017-08-21-15:07:04< jlledom> Another question: how do you send mails with git? git send-email is not supported in the hurd, maybe an old git version
*** poweroff commands.  I cannot use reboot.  I need to use reboot-hurd or poweroff-hurd to sync discs
hurd-2017-11-12-11:23:58< youpi> alternatively, you can use reboot-hurd and poweroff-hurd
hurd-2017-11-12-11:54:06< z3ntu> youpi: in a fresh install when running "poweroff-hurd" I get "In tight loop: hit ctrl-alt-del to reboot"

I can actually change the line "In tight loop: hit ctrl-alt-del to reboot" message to
"you can now safely turn your computer off"

Awesome that is a non-technical contribution that I can do!
*** a command can run as nobody
A process has can have very limited permissions.  A nobody process can only do the things you tell it to.
*** basic Hurd example of mach IPC with one process
http://hurdextras.nongnu.org/ipc_guide/mach_ipc_basic_concepts.html#1.1.1
** Mach
*** ports, port rights, and port names
A port is a message que.  You can think of a port as a small box full of envelopes.

A port set is a collection of message ques.  You can think of a port set as a desk that has many boxes on top of it.  A port set is useful in that a desk is useful in organizing many messages.

The port name is the name of the port. /mach_port_t/  ie: Benjamin.  It's just the name of the box of messages.  The port name points to an item in the tasks port name space.  What's the tasks port name space?  The item has either a dead name, a send-once right, or a combination of a receive right and a send right.
*** functions
*** paying interface
http://lists.gnu.org/archive/html/l4-hurd/2002-06/msg00001.html
*** a task is a thing that a micokernel needs to do
*** the things that Mach provides that makes it NOT a true nanokernel

http://darnassus.sceen.net/~hurd-web/microkernel/mach/concepts/

Mach is a first-generation microkernel.

Mach's basic abstractions include virtual address spaces in the form of tasks, execution contexts in the form of threads, IPC, capabilities in the form of ports, and memory objects, which enable Mach's external pager mechanism.

Controlling tasks, their virtual address space, threads, and other system objects in Mach is implemented by using ports, as opposed to other kernels' system call interface: almost all of the Mach API is implemented by sending messages to ports. Device drivers that reside in kernel space are controlled by ports, too.

Mach's ?API is well-documented.

In particular the [mach_kernel_principles] book further elaborates on Mach's concepts and principles.
IRC, freenode, #hurd, 2013-08-26

< stargater> then is mach not more microkernel
< stargater> when it have driver inside
< braunr> mach is a hybrid
< braunr> even without drivers
< stargater> in www i read mach is microkernel
< stargater> not hybrid
< braunr> the word microkernel usually includes hybrids
< braunr> true microkernels are also called nanokernels
< braunr> the word isn't that important, what matters is that mach does
  more in kernel than what the microkernel principle implies
*< braunr> e.g. high level async IPC and high level virtual memory operations*
*< braunr> including physical memory management*
*** may add messages without checking for integer overflows
hurd-2016-06-08-16:15:17< braunr> i just noticed that mach adds the overhead of messages without checking for integer overflows
*** inline function are better than macros
hurd-2017-01-19-16:15:46< braunr> avarzille: you should take the habit of using inline functions instead of macros as often as you can
*** gnumach regarding return values
hurd-2017-01-19-16:18:51< avarzille> I'm asking specifically for the sleeping lock
hurd-2017-01-19-16:18:58< avarzille> I use 0 (success) and -1 (error)
hurd-2017-01-19-16:21:51< braunr> kern_return_t
hurd-2017-01-19-16:22:56< avarzille> oh, I thought that was for communicating with userland
hurd-2017-01-19-16:23:20< braunr> i'd advise never using a boolean code, unless the function clearly returns a boolean, i.e. true or false
hurd-2017-01-19-16:23:31< braunr> no, it's also internal
hurd-2017-01-19-16:23:59< avarzille> ok
hurd-2017-01-19-16:24:16< braunr> the most common return types are void (of course), int (error code, here kern_return_t), bool (for condition tests), and void * (NULL meaning only possible error such as ENOMEM)
hurd-2017-01-19-16:24:21< braunr> people expect that
hurd-2017-01-19-16:24:30< braunr> using boolean for error codes is bound to cause confusion

*** a new strategy of OS design
**** the two servers in the Hurd that a special

The auth server.  This is the server that all programs trust to establish the identity of other servers.

The process server controls system components and does global booking operations.
**** translators
Thu Hurd uses mach ports to communicate info from users and servers.

So for example, when you open a file, the server owing that file node opens a port.  The user can then read the file by communicating to the port.  When a user normally opens the file, the filesystem opens a port to the file.  A filesystem would then jungle a bunch of ports.  For every open file, the filesystem would have an open port.

However a file could have a translator associated with it.  When a user accesses the file, the filesystem server then runs the translator program.  That translator program receives a port to the contents of the file, and returns a port to the user.  The translator then translates the underlying information in a different way.

But when a different user accesses that file, they see the original file.  Not the translator.

The xmlfs translator does this.  It translates an xml file into a directory tree.  Then one can cd into what used to be an xml file.  How cool is that!?

A translator could be used to untar a tar file.  This would perhaps be a bit slower, but also potentially convenient.
*** IPC
Mach's IPC allows syncronous and asyncronous communication.

It also allows the transmission of VERY large data.
**** mach_msg vs. mach_msg_trap

mach_msg_trap I believe is used when userspace stuff wants to send messages.

mach_msg is the kernel one...Both are essentially syscalls.
*** mach's vm vs. radixvm  ?
*** mach now uses x15's physical memory management?  irc log from 2015
hurd-2015-04-17:13:44:25< teythoon> i have integrated x15's physical memory management into gnumach and got it to compile while still using the old routines
*** gnu mach uses a slab allocater and an rbtree from x15 radix trees replaced ipc tables as well
*** RPC types should be fixed sized?
*** vm_page_allocator has been merged into gnumach from x15
hurd-2016-09-26:21:42:38< braunr> since early this year, i merged the vm_page allocator from x15 into gnumach
*** has general locks, braunr wants to replace them with
hurd-2016-11-02-14:47:54< braunr> and i'd like to see them gone and replaced with two new types, close to the posix ones
hurd-2016-11-02-14:47:58< braunr> "mutex" and "rwlock"
hurd-2016-11-02-14:48:26< braunr> the optimization i'm thinking about here is described in "futexes are tricky"

also youpi is an expert on concurrency
*** Mach has the ability to create mach servers.  The Hurd doesn't want to use these things!

There was some support for drivers in the kernel via mach servers, but this will hopefully be abandoned one day.
*** a big design mistake

GNUMach only has async IPC.  L4 only has sync IPC.  Clearly sync IPC was the better decision.
** genode and l4
https://www.youtube.com/user/drsartakov?feature=watch
** x15

X15 is Richard Braun's micorkernel.  Richard Braun is a veteran Hurd developer, who still occasionally chips into to help develop from time to time.  He believes that a total rewrite of the Hurd is the best way forward.  X15 is a new microkernel that reuses many ideas from Mach, while improving on others.  The Hurd developers occasionally can occasionally use some code from x15 inside GNU Mach, but x15 does not intend to replace GNU Mach.

*** rdxtrees are used with mutexes...
*** vm_page vs. mach's vm_page

The Hurd's vm_page uses a buddy allocator from x15 instead of a single list...Though I believe the buddy allocator was ported to gnumach.

a slab allocator is used for kernel memory allocations...
*** #x15 has its own irc channel
*** as of 2015 x15 does not have ipc
*** braunr wants true unix signals
hurd-2016-12-23-20:03:17< braunr> but i also want true unix signals
hurd-2016-12-23-20:03:26< braunr> well, true async userspace interrupts

** s16 is another microkernel?
hurd-2015-09-21-15:48:45< JX7P> this is why i wonder whether some of you folk may find S16 interesting
hurd-2015-09-21-15:49:02< braunr> but we could have something like singularity, no userspace, and we'd be fine as long as the interfaces and the isolation are actually there
hurd-2015-09-21-15:49:12< braunr> JX7P: definitely
hurd-2015-09-21:15:49:35< braunr> (i also find it interesting you named it s16, considering my own project is x15, but that's just a fun fact to notice)

** IDL
IDL stands for an interface definition language, and it is used to set up communication between servers and clients to facilitate remote procedure calls.

Braunr thinks that a microkernel should not have IDL.  Apparently the *best* way to do IPC is syncronous RPC shared, shared memory, signals, no IDL.
** TODO web website [1/7]
:LOGBOOK:
- State "TODO"       from              [2017-04-27 Thu 16:39]
:END:
*** TODO add to translator primer [2/3]
:LOGBOOK:
- State "TODO"       from              [2017-04-27 Thu 16:13]
:END:
**** DONE I can mention settrans ~/.signature /hurd/run /usr/games/fortune
CLOSED: [2017-04-27 Thu 16:39]
:LOGBOOK:
- State "DONE"       from "TODO"       [2017-04-27 Thu 16:39]
- State "TODO"       from              [2017-04-27 Thu 16:13]
:END:
**** DONE I need to change the number 3 to 4 on the page.
CLOSED: [2017-05-09 Tue 12:50]
:LOGBOOK:
- State "DONE"       from "TODO"       [2017-05-09 Tue 12:50]
- State "TODO"       from              [2017-04-27 Thu 16:44]
:END:
**** TODO I can mention more indepth pages at the bottom  I did this?
:LOGBOOK:
- State "TODO"       from              [2017-04-27 Thu 16:14]
:END:
https://www.gnu.org/software/hurd/hurd/translator/examples.html

Or this page
https://www.gnu.org/software/hurd/hurd/translator.html
*** TODO debugging subhurd I did this?
:LOGBOOK:
- State "TODO"       from              [2017-04-27 Thu 16:29]
:END:
The webpage says that you have to be root to start a subhurd

That's not true anymore
https://www.gnu.org/software/hurd/hurd/debugging/subhurd.html
*** TODO hurd/translator.html  I did this?
:LOGBOOK:
- State "TODO"       from              [2017-04-27 Thu 16:31]
:END:
This page does not mention that there is a python stubs for writing hurd translators
https://www.gnu.org/software/hurd/hurd/translator.html
*** Any documentation online that says "kvm" let it be known that qemu-system-i386 --enable-kvm is the same thing.
*** TODO debugging/rptrace.html
:LOGBOOK:
- State "TODO"       from              [2017-04-27 Thu 16:34]
:END:

Doesn't give an example of how to use it.
https://www.gnu.org/software/hurd/hurd/debugging/rpctrace.html
*** TODO mach print syscall
:LOGBOOK:
- State "TODO"       from              [2017-04-27 Thu 16:36]
:END:

There is no documentation about how to use this.
https://www.gnu.org/software/hurd/microkernel/mach/gnumach/interface/syscall/mach_print.html
*** TODO the debugging page doesn't have a portinfo specific page
:LOGBOOK:
- State "TODO"       from              [2017-04-27 Thu 16:39]
:END:
https://www.gnu.org/software/hurd/hurd/debugging.html
*** DONE explain how to check for translator options
CLOSED: [2017-05-09 Tue 13:13]
:LOGBOOK:
- State "DONE"       from "TODO"       [2017-05-09 Tue 13:13]
- State "TODO"       from              [2017-05-06 Sat 18:10]
:END:
ie:

#+BEGIN_SRC sh :results output :exports both :dir /ssh:joshua@localhost#2222:/home/joshua
/hurd/hello --help
#+END_SRC

#+RESULTS:
#+begin_example
Usage: hello [OPTION...]
A translator providing a warm greeting.

  -c, --contents=STRING      Specify the contents of the virtual file
  -?, --help                 Give this help list
      --usage                Give a short usage message
  -V, --version              Print program version

Mandatory or optional arguments to long options are also mandatory or optional
for any corresponding short options.

Report bugs to <bug-hurd@gnu.org>.
#+end_example
*** update hardware compatibility list
file:///home/joshua/programming/gnu/hurd/vm/web-page/home/joshua/programming/web.rendered/microkernel/mach/gnumach/hardware_compatibility_list.html
*** remove references of upstart

Upstart is not maintained.
*** new driver theory talks about DDE but not rump
https://www.gnu.org/software/hurd/community/gsoc/project_ideas/driver_glue_code.html
*** physical memory management page says that the Hurd's max memory is 800MB, but this is not true anymore.
https://www.gnu.org/software/hurd/community/gsoc/project_ideas/physical_memory_management.html
*** hurdish TCP/IP google summer of code stack not uses lwip not pfinit.  can we update this page
https://www.gnu.org/software/hurd/community/gsoc/project_ideas/tcp_ip_stack.html
*** gsoc sound support pulseaudio works on the hurd now...
https://www.gnu.org/software/hurd/community/gsoc/project_ideas/sound.html
*** remove Add UTIME_NOW and UTIME_OMIT.  Samuel apparently is about to finish this.
https://www.gnu.org/software/hurd/contributing.html#index1h2
** microkernel
*** what makes a microkernel fast?
small size?  http://darnassus.sceen.net/~hurd-web/microkernel/fud/  l4  different schedulers in userspace.
*** multiserver microkernelt
The microkernel only implements basic memory management, IPC.

Even the scheduler could run in userspace!  That's what l4 does.  This means that different processes could run on different schedulers!

The rest of the traditional kernel, TCP/IP, filesystems, device drivers, can all run in userspace as separate processes.  This means a normal user can modify things that would normally be in the kernel.

It's also possible to have a microkernel running, and then run one single userspace process for TCP/IP, filesystems, device drivers, etc.  That's essentially what mac does... I think.
** AHCI is normally needed for multicpu support?  but it is not for x15?  acpica does this in x15?
* things I don't understand
** pflocal?
file_name_lookup is related??  Hurd hacking guide 29

w/o this pipe 2 doesn't work

man 2 pipe

** /servers/socket/1  ?
** /servers/socket/2 is for the IPv4 stack
** /servers/socket/26 is for IPv6
** /dev/eth0 is DDE
** accessing pci cards as user

fsysopts /servers/pci --uid 1234 --p 00:1f.3

Allows a user to access a pci card?

Or I can configure it permanently via settrans
** /hurd/hostmux?
** remap
remap /bin/sh  $HOME/bin/sh

Now my scripts will remap /bin/sh to ~/bin/sh

remap /bin $HOME/unionbin

/bin will now look in ~/unionbin
** settrans -fga $HOME/servers/socket/2 /hurd/lwip -6 $HOME/servers/socket/26 -i $HOME/dev/tun0 -a 10.8.0.6 -m 255.255.255.0 -g 10.8.0.1

What does this do?

http://darnassus.sceen.net/~jlledom/en/09-point-to-point.html
** what is [[file:~/prog/gnu/hurd/hurd/TODO::**%20arla%20--%20port%20it][arla]] and why should it be ported?
* Things I could potentially write for
** the hurd
*** ext3fs
http://savannah.gnu.org/task/?5498

I've downloaded the patch to support ext3fs: [[file:~/Downloads/ext2fs_20050822.patch::console/][here.]]
*** libcap providing capabilities on the hurd
https://www.gnu.org/software/hurd/community/gsoc/project_ideas/libcap.html

I could write a dummy library first.
*** nfs and nfsd
*** finish the valgrind port
https://www.gnu.org/software/hurd/community/gsoc/project_ideas/valgrind.html
*** bcachefs
https://www.patreon.com/posts/status-update-11721750

90% of the code runs in userspace!  That means that bcachefs could be ported to the Hurd with little difficulty.
*** xmlfs doesn't update the directory structure when the file is saved.

But it does work!

settrans -ca xml /hurd/xmlfs example.xml

*** firewall translator
http://savannah.gnu.org/task/?12723
*** Make the Hurd console's configuration use xkb layout/variant instead of keymap.
http://xkbcommon.org/doc/current/index.html  could be used for
On the contributing page
https://www.gnu.org/software/hurd/contributing.html
*** multiprocessor support
http://savannah.gnu.org/task/?7179
*** random translator  isn't this done?
http://savannah.gnu.org/task/?5130
*** overwriting default servers
http://savannah.gnu.org/task/?6612
*** gtk+ driver
*** copy and paste support for console
http://savannah.gnu.org/task/?1427
*** ftpfs hangs
http://savannah.gnu.org/bugs/?24383
*** string_t is limited to 1024 bytes
http://savannah.gnu.org/bugs/?28511
*** make a /proc/cpuinfo
*** test driver development
https://www.gnu.org/software/hurd/community/gsoc/project_ideas/testing_framework.html
*** run glib's and other software's test-suites
https://www.gnu.org/software/hurd/community/gsoc/project_ideas/testsuites.html
*** pfinet
**** rewrite pfinet
http://savannah.gnu.org/task/?5469
There is a google summer of project for 2017 to do this.
**** pfinet hangs
http://savannah.gnu.org/bugs/?27539

*** NSF is pretty much crap
http://savannah.gnu.org/task/?5497
*** porting programs to the Hurd that fail because of PATH_MAX

https://www.gnu.org/software/hurd/community/gsoc/project_ideas/maxpath.html

currently abiword isn't working on the hurd because of this.

*** trying to extend glibc for the Hurd
https://www.gnu.org/software/hurd/community/gsoc/project_ideas/testsuites.html

https://www.gnu.org/software/hurd/source_repositories/glibc.html

https://www.gnu.org/software/hurd/open_issues/glibc.html#index4h2

https://www.gnu.org/software/hurd/glibc.html

https://www.gnu.org/software/hurd/open_issues/glibc.html

https://www.gnu.org/software/hurd/glibc/mmap.html

moving mount/umount into libc is a easy hacking thing to do apparently.
*** modify programs to use the hurd translaters
https://www.gnu.org/software/hurd/community/gsoc/project_ideas/download_backends.html
*** automake'ify hurd's code base
http://savannah.gnu.org/task/?5491
*** add an input driver for console client
http://savannah.gnu.org/task/?2503

** documentation things I can fix
*** Hurd hacking guide
**** page 34
Why should I learn about Mach if the Hurd will switch to L4 soon?
*** hurd.ps
**** TODO "3.4 Misc Library" There is a hurd library called "libshouldbeinlibc".  Some of those functions are added into the glibc.
:LOGBOOK:
- State "TODO"       from              [2016-06-02 Thu 07:18]
:END:
I can document any new functions that are added in the GNU C Library Reference Manual.
**** TODO 5.1 Translators  FIXME We need a good example
:LOGBOOK:
- State "TODO"       from              [2016-06-02 Thu 07:25]
:END:
**** TODO 5.1.3 Invoking mount
:LOGBOOK:
- State "TODO"       from              [2016-06-02 Thu 07:37]
:END:
**** TODO 5.4.8 Notifications has nothing in it
:LOGBOOK:
- State "TODO"       from              [2016-06-02 Thu 07:44]
:END:
**** TODO 5.4.9 File Translators Fixme how to set and get translators
:LOGBOOK:
- State "TODO"       from              [2016-06-02 Thu 07:45]
:END:
**** TODO 5.5 Filesystem interface
:LOGBOOK:
- State "TODO"       from              [2016-06-02 Thu 07:45]
:END:
**** TODO 8 Stored Filesystems has almost no information
:LOGBOOK:
- State "TODO"       from              [2016-06-02 Thu 07:47]
:END:
**** TODO 8.2 Linux Extended 2 FS
:LOGBOOK:
- State "TODO"       from              [2016-06-02 Thu 07:48]
:END:
This has no information.  It should possibly explain why ext3/4 is not implemented in the Hurd.
**** TODO 9 Twisted Filesystems
:LOGBOOK:
- State "TODO"       from              [2016-06-02 Thu 07:49]
:END:
This has almost on information as well.
**** TODO 10 Distributed Filesystems
:LOGBOOK:
- State "TODO"       from              [2016-06-02 Thu 07:50]
:END:
Has almost no information.
**** TODO 11 Networking
:LOGBOOK:
- State "TODO"       from              [2016-06-02 Thu 07:54]
:END:
Has almost no information
**** TODO 12 Terminal handling
:LOGBOOK:
- State "TODO"       from              [2016-06-02 Thu 07:55]
:END:
**** TODO 13 Has no information
:LOGBOOK:
- State "TODO"       from              [2016-06-02 Thu 07:55]
:END:
**** TODO 14 has no information
:LOGBOOK:
- State "TODO"       from              [2016-06-02 Thu 07:55]
:END:
*** things I finished today

**** DONE "3.1 Threads Library" claims that the Hurd uses cthreads, when it actually uses pthreads
CLOSED: [2017-04-22 Sat 17:20]
:LOGBOOK:
- State "DONE"       from "TODO"       [2017-04-22 Sat 17:20]
- State "TODO"       from "TODO"       [2017-04-22 Sat 17:19]
- State "TODO"       from              [2016-06-02 Thu 07:07]
:END:
**** DONE it mentions libtreefs should be talked about in the [[file:/ssh:joshua@localhost#2222:/home/joshua/programming/hurd/doc/hurd.texi::FIXME:%20a%20chapter%20on%20libtreefs%20and%20libdirmgt%20will%20probably%20go%20here][hurd.ps]]
CLOSED: [2017-04-22 Sat 17:23]
:LOGBOOK:
- State "DONE"       from "TODO"       [2017-04-22 Sat 17:23]
- State "TODO"       from              [2017-04-22 Sat 17:23]
:END:
BUT the hacking guide says that "libtreefs is defunct.  It was never finished.  Nobody uses it.
Neither should you."

https://www.gnu.org/software/hurd/hacking-guide/hhg.txt
**** DONE added a reference to [[file:/ssh:joshua@localhost#2222:/home/joshua/programming/hurd/doc/hurd.texi::Single%20files%20with%20dynamic%20content,%20such%20as%20/hurd/random.][/hurd/random]]
CLOSED: [2017-04-22 Sat 17:32]
:LOGBOOK:
- State "DONE"       from              [2017-04-22 Sat 17:32]
:END:
**** DONE I added some detail about how to use the [[file:/ssh:joshua@localhost#2222:/home/joshua/programming/hurd/doc/hurd.texi::The%20@code{mount}%20program%20has%20the%20following%20synopsis:][mount]] program.  I just copied the output of mount --help.
CLOSED: [2017-04-22 Sat 17:51]
:LOGBOOK:
- State "DONE"       from              [2017-04-22 Sat 17:51]
:END:
** gnumach
*** [#A] readahead
*** [#A] multipage paging
*** let mach create futexes
http://savannah.gnu.org/task/?6231
*** rewrite mach's IPC and VM
http://savannah.gnu.org/task/?5489
*** replace many of the simple spin locks with mutexes?
braunr said on irc

that there are very few reasons that one would use spin locks.  These are not very fast.

He wants to replace the "general locks" with something similar to POSIX mutex and rwlock.  I can read more about it in the article "futexes are tricky".

gsync_requeue has an implementation & braunr has an implementation as well in x15.

youpi thinks we should use wait queues...

*** rewrite gnu mach device driver glue code
http://savannah.gnu.org/task/?5488
*** lots of stuff on this page:

[[file:~/programming/gnu/hurd/hurd-web/microkernel/mach/gnumach/projects.mdwn][file:~/programming/gnu/hurd/hurd-web/microkernel/mach/gnumach/projects.mdwn]]

*** mach 5 IPC communication
file:///home/joshua/programming/gnu/hurd/hurd-web.rendered/microkernel/mach/gnumach/projects/mach_5.html
*** clean up the mach code
file:///home/joshua/programming/gnu/hurd/hurd-web.rendered/microkernel/mach/gnumach/projects/clean_up_the_code.html
*** gnumach uses a lot of spin locks.  Brauner thinks they should be replaced with mutexes.
Then mach should provide futexes.

http://lists.gnu.org/archive/html/bug-hurd/2012-12/msg00002.html
** porting packages
file:///home/joshua/programming/gnu/hurd/hurd-web.rendered/community/weblogs/ArneBab/porting-simple-packages.html
* bug notes
:PROPERTIES:
:ID:       87a96c40-ab25-4c53-8e31-cb03dfdc4677
:END:
** mach
*** mach compile errors
****  linux/dev/init/main.c: In function 'alloc_contig_mem':
                 linux/dev/init/main.c:299:3: warning: return makes pointer from integer without
                 a cast
                    return phystokv(m);
                    ^
****   linux/dev/init/main.c: In function 'alloc_contig_mem':
                 linux/dev/init/main.c:196:15: warning: 'addr' may be used uninitialized in this
                 function [-Wmaybe-uninitialized]
                    vm_offset_t addr, max_addr;
****   linux/dev/glue/block.c: In function 'device_open':
                 linux/dev/glue/block.c:1160:14: warning: assignment from incompatible pointer ty
                 pe
                         *devp = IP_NULL;
                               ^
****   linux/dev/glue/block.c: In function 'device_get_status':
                 linux/dev/glue/block.c:1669:28: warning: initialization from incompatible pointe
                 r type
                     struct disk_parms *dp = status;
****  linux/dev/drivers/block/floppy.c: In function 'fd_ioctl':
                 linux/dev/drivers/block/floppy.c:3314:32: warning: pointer targets in passing ar
                 gument 1 of 'normalize_0x02xx_ioctl' differ in signedness [-Wpointer-sign]
                    ECALL(normalize_0x02xx_ioctl(&cmd, &size));
****   In file included from linux/dev/drivers/block/ahci.c:32:0:
                 ./linux/dev/include/linux/blk.h:167:21: warning: 'do_sd' defined but not used [-
                 Wunused-variable]
                  #define DEVICE_INTR do_sd
**** linux/src/drivers/block/triton.c: In function 'config_drive_for_dma':
                 linux/src/drivers/block/triton.c:223:24: warning: pointer targets in passing arg
                 ument 2 of 'strcmp' differ in signedness [-Wpointer-sign]
                     if (!strcmp(*list++,id->model)) {
****             In file included from ./linux/src/include/linux/string.h:47:0,
                                  from ./linux/src/include/asm/termios.h:59,
                                  from ./linux/src/include/linux/termios.h:5,
                                  from ./linux/src/include/linux/tty.h:20,
                                  from ./linux/dev/include/linux/sched.h:26,
                                  from ./linux/dev/include/linux/mm.h:4,
                                  from linux/src/drivers/block/triton.c:22:
                 ./linux/dev/include/asm/string.h:108:19: note: expected 'const char *' but argum
                 ent is of type 'unsigned char *'
                  static inline int strcmp(const char * cs,const char * ct)
****     linux/src/drivers/block/triton.c:253:24: warning: pointer targets in passing arg
                 ument 2 of 'strcmp' differ in signedness [-Wpointer-sign]
                     if (!strcmp(*list++,id->model)) {
                                         ^
****    In file included from ./linux/src/include/linux/string.h:47:0,
                                  from ./linux/src/include/asm/termios.h:59,
                                  from ./linux/src/include/linux/termios.h:5,
                                  from ./linux/src/include/linux/tty.h:20,
                                  from ./linux/dev/include/linux/sched.h:26,
                                  from ./linux/dev/include/linux/mm.h:4,
                                  from linux/src/drivers/block/triton.c:22:
                 ./linux/dev/include/asm/string.h:108:19: note: expected 'const char *' but argum
                 ent is of type 'unsigned char *'
                  static inline int strcmp(const char * cs,const char * ct)
****                 In file included from linux/src/drivers/scsi/hosts.c:129:0:
                 linux/src/drivers/scsi/dtc.h:49:0: warning: "CSR_RESET" redefined
                  #define CSR_RESET              0x80    /* wo  Resets 53c400 */
                  ^
                 In file included from linux/src/drivers/scsi/hosts.c:105:0:
                 linux/src/drivers/scsi/in2000.h:187:0: note: this is the location of the previou
                 s definition
                  #define CSR_RESET    0x00
****                  In file included from linux/src/drivers/scsi/hosts.c:141:0:
                 linux/src/drivers/scsi/ncr53c8xx.h:129:0: warning: "SCSI_NCR_SETUP_ULTRA_SCSI" r
                 edefined
                  #define SCSI_NCR_SETUP_ULTRA_SCSI  (2)
                  ^
                 In file included from linux/src/drivers/scsi/sym53c8xx.h:60:0,
                                  from linux/src/drivers/scsi/hosts.c:137:
                 linux/src/drivers/scsi/sym53c8xx_defs.h:154:0: note: this is the location of the
                  previous definition
                  #define SCSI_NCR_SETUP_ULTRA_SCSI  (3)
****                  In file included from linux/src/drivers/scsi/hosts.c:141:0:
                 linux/src/drivers/scsi/ncr53c8xx.h:129:0: warning: "SCSI_NCR_SETUP_ULTRA_SCSI" r
                 edefined
                  #define SCSI_NCR_SETUP_ULTRA_SCSI  (2)
                  ^
                 In file included from linux/src/drivers/scsi/sym53c8xx.h:60:0,
                                  from linux/src/drivers/scsi/hosts.c:137:
                 linux/src/drivers/scsi/sym53c8xx_defs.h:154:0: note: this is the location of the
                  previous definition
                  #define SCSI_NCR_SETUP_ULTRA_SCSI  (3)
**** quite a few more redefined defs
****             In file included from linux/src/drivers/scsi/hosts.c:117:0:
                 linux/src/drivers/scsi/qlogicisp.h:72:30: warning: 'proc_scsi_isp1020' defined b
                 ut not used [-Wunused-variable]
                  static struct proc_dir_entry proc_scsi_isp1020;
****            linux/src/drivers/scsi/scsi.c: In function 'scan_scsis':
                 linux/src/drivers/scsi/scsi.c:454:32: warning: pointer targets in passing argume
                 nt 9 of 'scan_scsis_single' differ in signedness [-Wpointer-sign]
                           &SDpnt, SCpnt, shpnt, scsi_result);
****                  linux/src/drivers/scsi/scsi.c:95:12: note: expected 'char *' but argument is of
                 type 'unsigned char *'
                  static int scan_scsis_single (int channel,int dev,int lun,int * max_scsi_dev ,
****                  linux/src/drivers/scsi/scsi.c:491:9: warning: pointer targets in passing argumen
                 t 9 of 'scan_scsis_single' differ in signedness [-Wpointer-sign]
                          scsi_result)
****                  linux/src/drivers/scsi/scsi.c:95:12: note: expected 'char *' but argument is of
                 type 'unsigned char *'
                  static int scan_scsis_single (int channel,int dev,int lun,int * max_scsi_dev ,
****             linux/src/drivers/scsi/scsi.c:525:23: warning: pointer targets in passing argume
                 nt 1 of 'scsi_init_free' differ in signedness [-Wpointer-sign]
                        scsi_init_free (scsi_result, 512);
****                In file included from linux/src/drivers/scsi/scsi.c:57:0:
                 linux/src/drivers/scsi/hosts.h:327:13: note: expected 'char *' but argument is o
                 f type 'unsigned char *'
                  extern void scsi_init_free(char * ptr, unsigned int size);
****             linux/src/drivers/scsi/scsi.c: In function 'scan_scsis_single':  LOTS HERE
                 linux/src/drivers/scsi/scsi.c:708:18: warning: pointer targets in passing argume
                 nt 1 of 'print_inquiry' differ in signedness [-Wpointer-sign]
                    print_inquiry (scsi_result);
                                   ^
                 linux/src/drivers/scsi/scsi.c:93:13: note: expected 'unsigned char *' but argume
                 nt is of type 'char *'
                  static void print_inquiry(unsigned char *data);
                              ^
                 linux/src/drivers/scsi/scsi.c:731:30: warning: pointer targets in passing argume
                 nt 1 of 'get_device_flags' differ in signedness [-Wpointer-sign]
                    bflags = get_device_flags (scsi_result);
                                               ^
                 linux/src/drivers/scsi/scsi.c:303:12: note: expected 'unsigned char *' but argum
                 ent is of type 'char *'
                  static int get_device_flags(unsigned char * response_data){
                             ^
                 linux/src/drivers/scsi/scsi.c: In function 'resize_dma_pool':
                 linux/src/drivers/scsi/scsi.c:2898:21: warning: pointer targets in passing argum
                 ent 1 of 'scsi_init_free' differ in signedness [-Wpointer-sign]
                       scsi_init_free(dma_malloc_pages[i], PAGE_SIZE);

                  linux/src/drivers/scsi/scsi.c:2513:6: note: expected 'char *' but argument is of
                  type 'unsigned char *'
                  void scsi_init_free(char * ptr, unsigned int size)

****             linux/src/drivers/scsi/scsicam.c: In function 'scsicam_bios_param':
                 linux/src/drivers/scsi/scsicam.c:58:5: warning: implicit declaration of function
                  'scsi_partsize' [-Wimplicit-function-declaration]
                      ret_code = scsi_partsize (bh, (unsigned long) size, (unsigned int *) ip + 2
                 ,
                      ^
**** 2        linux/src/drivers/scsi/sd.c: In function 'revalidate_scsidisk':
                 linux/src/drivers/scsi/sd.c:1551:9: warning: unused variable 'devi' [-Wunused-va
                 riable]
                   kdev_t devi = MKDEV(MAJOR_NR, minor);
                          ^
                 linux/src/drivers/scsi/sd.c: In function 'sd_detach':
                 linux/src/drivers/scsi/sd.c:1597:10: warning: unused variable 'devi' [-Wunused-v
                 ariable]
                    kdev_t devi = MKDEV(MAJOR_NR, minor);
                           ^
                 In file included from ./linux/dev/include/linux/blk.h:6:0,
                                  from linux/src/drivers/scsi/sd.c:42:
****           linux/src/drivers/scsi/sr.c: In function 'sr_detach':
                 linux/src/drivers/scsi/sr.c:1218:13: warning: unused variable 'devi' [-Wunused-v
                 ariable]
                       kdev_t devi = MKDEV(MAJOR_NR, i);
**** 6  linux/src/drivers/scsi/sr_ioctl.c: In function 'sr_ioctl':
                 linux/src/drivers/scsi/sr_ioctl.c:219:9: warning: pointer targets in assignment
                 differ in signedness [-Wpointer-sign]
                   buffer = (unsigned char *) scsi_malloc(512);
                          ^
                 linux/src/drivers/scsi/sr_ioctl.c:361:9: warning: pointer targets in assignment
                 differ in signedness [-Wpointer-sign]
                   buffer = (unsigned char *) scsi_malloc(512);
                          ^
                 linux/src/drivers/scsi/sr_ioctl.c:377:7: warning: pointer targets in assignment
                 differ in signedness [-Wpointer-sign]
                   mask = (unsigned char *) scsi_malloc(512);
                        ^
                 linux/src/drivers/scsi/sr_ioctl.c:427:9: warning: pointer targets in assignment
                 differ in signedness [-Wpointer-sign]
                   buffer = (unsigned char *) scsi_malloc(512);
                          ^
                 linux/src/drivers/scsi/sr_ioctl.c:463:9: warning: pointer targets in assignment
                 differ in signedness [-Wpointer-sign]
                   buffer = (unsigned char*) scsi_malloc(512);
                          ^
                 linux/src/drivers/scsi/sr_ioctl.c:505:9: warning: pointer targets in assignment
                 differ in signedness [-Wpointer-sign]
                   buffer = (unsigned char*) scsi_malloc(512);

****        In file included from linux/src/drivers/scsi/AM53C974.c:18:0:
                 linux/src/drivers/scsi/AM53C974.c: In function 'AM53C974_sync_neg':
                 linux/src/drivers/scsi/AM53C974.h:291:48: warning: variable 'io_port' set but no
                 t used [-Wunused-but-set-variable]
                  #define AM53C974_local_declare() unsigned long io_port
**** 2        linux/src/drivers/scsi/AM53C974.c: In function 'AM53C974_information_transfer':
                 linux/src/drivers/scsi/AM53C974.c:1422:21: warning: pointer targets in assignmen
                 t differ in signedness [-Wpointer-sign]
                         cmd->SCp.ptr = (unsigned char *)cmd->SCp.buffer->address;
                                      ^
                 linux/src/drivers/scsi/AM53C974.c:1398:26: warning: variable 'ret' set but not u
                 sed [-Wunused-but-set-variable]
                  int                      ret, i, len, residual=-1;
****              In file included from linux/src/drivers/scsi/BusLogic.c:53:0:
                 linux/src/drivers/scsi/FlashPoint.c: In function 'FlashPoint_AbortCCB':
                 linux/src/drivers/scsi/FlashPoint.c:4509:8: warning: variable 'TID' set but not
                 used [-Wunused-but-set-variable]
                   UCHAR TID;
                         ^
                 linux/src/drivers/scsi/BusLogic.c: In function 'BusLogic_DriverInfo':
                 linux/src/drivers/scsi/BusLogic.c:176:3: warning: pointer targets in return diff
                 er in signedness [-Wpointer-sign]
                    return HostAdapter->FullModelName;
                    ^
                 linux/src/drivers/scsi/BusLogic.c: In function 'BusLogic_ReadHostAdapterConfigur
                 ation':
                 linux/src/drivers/scsi/BusLogic.c:1519:14: warning: pointer targets in passing a
                 rgument 1 of 'strcpy' differ in signedness [-Wpointer-sign]
                        strcpy(HostAdapter->FirmwareVersion, FlashPoint_FirmwareVersion);

                 In file included from ./linux/src/include/linux/string.h:47:0,
                                  from ./linux/src/include/asm/termios.h:59,
                                  from ./linux/src/include/linux/termios.h:5,
                                  from ./linux/src/include/linux/tty.h:20,
                                  from ./linux/dev/include/linux/sched.h:26,
                                  from ./linux/dev/include/linux/blkdev.h:5,
                                  from linux/src/drivers/scsi/BusLogic.c:37:
                 ./linux/dev/include/asm/string.h:31:22: note: expected 'char *' but argument is
                 of type 'unsigned char *'
                  static inline char * strcpy(char * dest,const char *src)
                                       ^
                 linux/src/drivers/scsi/BusLogic.c:1600:12: warning: pointer targets in passing a
                 rgument 1 of 'strcpy' differ in signedness [-Wpointer-sign]
                      strcpy(HostAdapterModelNumber, "542B");

                  In file included from ./linux/src/include/linux/string.h:47:0,
                                  from ./linux/src/include/asm/termios.h:59,
                                  from ./linux/src/include/linux/termios.h:5,
                                  from ./linux/src/include/linux/tty.h:20,
                                  from ./linux/dev/include/linux/sched.h:26,
                                  from ./linux/dev/include/linux/blkdev.h:5,
                                  from linux/src/drivers/scsi/BusLogic.c:37:
                 ./linux/dev/include/asm/string.h:31:22: note: expected 'char *' but argument is
                 of type 'unsigned char *'
                  static inline char * strcpy(char * dest,const char *src)
                                       ^
                 linux/src/drivers/scsi/BusLogic.c:1607:12: warning: pointer targets in passing a
                 rgument 1 of 'strcpy' differ in signedness [-Wpointer-sign]
                      strcpy(HostAdapterModelNumber, "742A");

                 In file included from ./linux/src/include/linux/string.h:47:0,
                                  from ./linux/src/include/asm/termios.h:59,
                                  from ./linux/src/include/linux/termios.h:5,
                                  from ./linux/src/include/linux/tty.h:20,
                                  from ./linux/dev/include/linux/sched.h:26,
                                  from ./linux/dev/include/linux/blkdev.h:5,
                                  from linux/src/drivers/scsi/BusLogic.c:37:
                 ./linux/dev/include/asm/string.h:31:22: note: expected 'char *' but argument is
                 of type 'unsigned char *'
                  static inline char * strcpy(char * dest,const char *src)
                                       ^
                 linux/src/drivers/scsi/BusLogic.c:1611:12: warning: pointer targets in passing a
                 rgument 1 of 'strcpy' differ in signedness [-Wpointer-sign]
                      strcpy(HostAdapterModelNumber, "747A");

                       In file included from ./linux/src/include/linux/string.h:47:0,
                                  from ./linux/src/include/asm/termios.h:59,
                                  from ./linux/src/include/linux/termios.h:5,
                                  from ./linux/src/include/linux/tty.h:20,
                                  from ./linux/dev/include/linux/sched.h:26,
                                  from ./linux/dev/include/linux/blkdev.h:5,
                                  from linux/src/drivers/scsi/BusLogic.c:37:
                 ./linux/dev/include/asm/string.h:31:22: note: expected 'char *' but argument is
                 of type 'unsigned char *'
                  static inline char * strcpy(char * dest,const char *src)
                                       ^
                 linux/src/drivers/scsi/BusLogic.c:1665:14: warning: pointer targets in passing a
                 rgument 1 of 'strcmp' differ in signedness [-Wpointer-sign]
                    if (strcmp(HostAdapter->FirmwareVersion, "3.3") >= 0)

 In file included from ./linux/src/include/linux/string.h:47:0,
                                  from ./linux/src/include/asm/termios.h:59,
                                  from ./linux/src/include/linux/termios.h:5,
                                  from ./linux/src/include/linux/tty.h:20,
                                  from ./linux/dev/include/linux/sched.h:26,
                                  from ./linux/dev/include/linux/blkdev.h:5,
                                  from linux/src/drivers/scsi/BusLogic.c:37:
                 ./linux/dev/include/asm/string.h:108:19: note: expected 'const char *' but argum
                 ent is of type 'unsigned char *'
                  static inline int strcmp(const char * cs,const char * ct)
                                    ^
                 linux/src/drivers/scsi/BusLogic.c:1829:19: warning: pointer targets in passing a
                 rgument 1 of 'strcmp' differ in signedness [-Wpointer-sign]
                         if (strcmp(HostAdapter->ModelName, "BT-757") == 0)

                In file included from ./linux/src/include/linux/string.h:47:0,
                                  from ./linux/src/include/asm/termios.h:59,
                                  from ./linux/src/include/linux/termios.h:5,
                                  from ./linux/src/include/linux/tty.h:20,
                                  from ./linux/dev/include/linux/sched.h:26,
                                  from ./linux/dev/include/linux/blkdev.h:5,
                                  from linux/src/drivers/scsi/BusLogic.c:37:
                 ./linux/dev/include/asm/string.h:108:19: note: expected 'const char *' but argum
                 ent is of type 'unsigned char *'
                  static inline int strcmp(const char * cs,const char * ct)
                                    ^
                 linux/src/drivers/scsi/BusLogic.c:1873:14: warning: pointer targets in passing a
                 rgument 1 of 'strcmp' differ in signedness [-Wpointer-sign]
                    if (strcmp(HostAdapter->FirmwareVersion, "3.31") >= 0)

                     In file included from ./linux/src/include/linux/string.h:47:0,
                                  from ./linux/src/include/asm/termios.h:59,
                                  from ./linux/src/include/linux/termios.h:5,
                                  from ./linux/src/include/linux/tty.h:20,
                                  from ./linux/dev/include/linux/sched.h:26,
                                  from ./linux/dev/include/linux/blkdev.h:5,
                                  from linux/src/drivers/scsi/BusLogic.c:37:
                 ./linux/dev/include/asm/string.h:108:19: note: expected 'const char *' but argum
                 ent is of type 'unsigned char *'
                  static inline int strcmp(const char * cs,const char * ct)
                                    ^
                 linux/src/drivers/scsi/BusLogic.c:1899:18: warning: pointer targets in passing a
                 rgument 1 of 'strcmp' differ in signedness [-Wpointer-sign]
                        if (strcmp(HostAdapter->FirmwareVersion, "4.22") >= 0)
                                   ^

                                   In file included from ./linux/src/include/linux/string.h:47:0,
                                  from ./linux/src/include/asm/termios.h:59,
                                  from ./linux/src/include/linux/termios.h:5,
                                  from ./linux/src/include/linux/tty.h:20,
                                  from ./linux/dev/include/linux/sched.h:26,
                                  from ./linux/dev/include/linux/blkdev.h:5,
                                  from linux/src/drivers/scsi/BusLogic.c:37:
                 ./linux/dev/include/asm/string.h:108:19: note: expected 'const char *' but argum
                 ent is of type 'unsigned char *'
                  static inline int strcmp(const char * cs,const char * ct)
                                    ^
                 linux/src/drivers/scsi/BusLogic.c:1903:18: warning: pointer targets in passing a
                 rgument 1 of 'strcmp' differ in signedness [-Wpointer-sign]
                        if (strcmp(HostAdapter->FirmwareVersion, "3.35") >= 0)

                         In file included from ./linux/src/include/linux/string.h:47:0,
                                  from ./linux/src/include/asm/termios.h:59,
                                  from ./linux/src/include/linux/termios.h:5,
                                  from ./linux/src/include/linux/tty.h:20,
                                  from ./linux/dev/include/linux/sched.h:26,
                                  from ./linux/dev/include/linux/blkdev.h:5,
                                  from linux/src/drivers/scsi/BusLogic.c:37:
                 ./linux/dev/include/asm/string.h:108:19: note: expected 'const char *' but argum
                 ent is of type 'unsigned char *'
                  static inline int strcmp(const char * cs,const char * ct)
                                    ^
                 linux/src/drivers/scsi/BusLogic.c:1928:14: warning: pointer targets in passing a
                 rgument 1 of 'strcmp' differ in signedness [-Wpointer-sign]
                        strcmp(HostAdapter->ModelName, "BT-445S") == 0 &&

*** Gnumach crashes after rpctrace exists unexpectedly
*** string_t is limited to 1024 bytes
*** gnumach doesn't check device name
http://savannah.gnu.org/bugs/?15300
*** TODO I should add an INSTALL file into gnumach.  it'll say

    autoreconf -fi
    ./configure
    make
    make install

    [[file:~/programming/org/gtd/projects/become-an-awesome-hacker.org::*Hurd][Hurd]]
*** TODO I can improve the mach manual by typing a few lines about Mach's history.  That would be helpful.  I can probably
     copy it from the wiki online.

     [[file:~/programming/org/gtd/projects/become-an-awesome-hacker.org::*Hurd][Hurd]]

Actually I cannot copy from that page.  That wiki page doesn't have the you can copy me section down toward the bottom of the webpage.

** MIG
*** MiG generated code does not destroy invalid reply DOS attacks
http://savannah.gnu.org/bugs/?48456
*** MIG fails to check un-terminated strings
http://savannah.gnu.org/bugs/?28446
** Hurd bugs
:PROPERTIES:
:ARCHIVE:  %s_archive::"*Hurd bugs"
:END:
*** un-initialized variable in init.c
http://savannah.gnu.org/bugs/?17129
*** terminal supports making lowercase letters on keyboards that only support uppercase
https://savannah.gnu.org/bugs/?48464
*** cannot replace translator is this really a bug?
#+BEGIN_SRC sh :results output :exports both
$ ls -ld hop
ls: cannot access 'hop': No such file or directory
$ touch hop
$ settrans hop /hurd/fifo
$ settrans hop /hurd/hello
$ showtrans hop
/hurd/fifo
#+END_SRC

#+BEGIN_SRC sh :results output :exports both
$ ls -ld hop
ls: cannot access 'hop': No such file or directory
$ touch hop
$ settrans hop /hurd/fifo
$ settrans -gf hop
$ settrans hop /hurd/hello
$ showtrans hop
/hurd/hello
#+END_SRC

Is this really a bug?

http://savannah.gnu.org/bugs/?48438
*** #49730 /proc/PID/environ returns I/O errors on read attempts
I'm having problems retrieving process environment from /proc. Something like "cat /proc/936/environ" often (but not always) returns an I/O error, while "msgport --getenv -p 936" works fine. I'm not sure exactly which processes suffer from this, but translators seem particularly vulnerable.

# I don't see any input output errors?

Studying the problem with rpctrace, it seems that there are two different ways to obtain a process's environment. The proc server hold a pointer to the environment array (in the process's address space, not its own), and then vm_read's it to answer proc_procgetenv RPCs. This is how procfs does it, and this is what isn't working.

I've written a short program (attached) to fetch the argument locations and print them. Comparing this output to /proc/PID/maps shows that on the affected processes, the addresses appear sane, but do not correspond to mapped memory locations, thus the I/O errors.

The other way to get a process's environment is to fetch the process's msg port and query it using a msg_get_environment RPC. The msgport program does this, and is able to successfully retrieve environment information.

Related: Do we need two different ways to retrieve a process's environment?

*** Hurd FIFO discarding options
http://savannah.gnu.org/bugs/?48437

#+BEGIN_SRC sh :results output :exports both
$ ls -ld hop
ls: cannot access 'hop': No such file or directory
$ touch hop
$ settrans hop /hurd/fifo --multiple-readers
$ showtrans hop
/hurd/fifo
#+END_SRC

The output from showtrans should have been "/hurd/fifo --multiple-readers".

Really though?
*** limits are not set to children

http://savannah.gnu.org/bugs/?43320

#+BEGIN_SRC sh :results output :exports both
$ ulimit -s
unlimited
$ ulimit -s 1024
$ ulimit -s
1024
$ bash
$ ulimit -s
unlimited
#+END_SRC
*** device_clone not called on port closure
http://savannah.gnu.org/bugs/?30256
*** ext2fs leaks memory on exec calls
http://savannah.gnu.org/bugs/?30096

Braunr may have fixed this bug.  I should try taking a look at the code on the website and seeing if running that code does indeed cause a memory leak.

*** translators output garbage with settrans -P and stopped by gdb
*** application bugs
**** gdb breakpoints in threads make SIGTRAP
http://savannah.gnu.org/bugs/?29642
**** git commit with Emacs -nw fails

http://savannah.gnu.org/bugs/?48864

When running 'git commit' with environment variable EDITOR set to 'emacs -nw', emacs works correctly to edit the commit message unless CNTL-Z is used to suspend the session. The shell prompt appears correctly, but when 'fg' is used to resume the git/commit, emacs does not redisplay. CNTL-Z can be used again to get back to the shell, but ultimately the 'git commit' has to be killed (with 'kill').
*** firm-link look results in memory exhaustion
http://savannah.gnu.org/bugs/?29809
*** ftpfs hangs if the network is down
http://savannah.gnu.org/bugs/?24383
*** libstore segfaults
A patch claims to fix it.
http://savannah.gnu.org/bugs/?20906
*** time returns bogus time
http://savannah.gnu.org/bugs/?20683
*** sudo resets the environmental variables
http://savannah.gnu.org/bugs/?20060
*** setauth causes a crash
http://savannah.gnu.org/bugs/?19767
*** nice values error.  already fixed?
http://savannah.gnu.org/bugs/?18883
*** glibc is missing SIGRTMIN and SIGRTMAX
http://savannah.gnu.org/bugs/?17650
*** glibc missing TLS
http://savannah.gnu.org/bugs/?17644

It's possible already fixed?
*** hurd/crash --suspend fails
http://savannah.gnu.org/bugs/?17319
*** resolution in translator issues
http://savannah.gnu.org/bugs/?17133
*** gnumach doesn't share IRQ with devices
http://savannah.gnu.org/bugs/?15355
*** proc leaks memory
http://savannah.gnu.org/bugs/?15322
*** verifying passwords is UID based
http://savannah.gnu.org/bugs/?15320
*** #6613 Hurd duplicates common GNU programs
http://savannah.gnu.org/task/?6613
*** dd uses up memory
http://savannah.gnu.org/bugs/?15297
*** #6612 over ride default servers
http://savannah.gnu.org/task/?6612
*** fs crashes a subhurd
http://savannah.gnu.org/bugs/?15317

Can this be reproduced?
*** /hurd/crash --suspend doesn't work.  I need a program that crashes...

Well I have one now.

#+BEGIN_SRC sh :results output :exports both :dir /ssh:joshua@localhost#2222:/home/joshua/
./crash
#+END_SRC

#+RESULTS:
: Segfault

But I don't understand if that means the bug is still broken?
http://savannah.gnu.org/bugs/?17319
** Debian bugs
https://bugs.debian.org/cgi-bin/pkgreport.cgi?package=hurd#_0_2_4
** small hack entries
https://www.gnu.org/software/hurd/contributing.html#index1h2

Small hack entries

Here is a list of small hacks, which can serve as entries into the Hurd code for people who would like to dive into the code but just lack a "somewhere to begin with".

*** Add UTIME_NOW and UTIME_OMIT. It is a matter of taking the BSD values, add the #defines to the proper header, and implement the support in *_S_file_utimes functions. See also Debian bug #762677.
*** Some translators do not support fsysopts, i.e. support for the file_get_fs_options and fsys_set_options RPCs.
*** Extend device_read/device_write into supporting > 2TiB disk sizes.
*** Make the Hurd console's configuration use xkb layout/variant instead of keymap.
*** Add NX protection support to GNU Mach.
*** Write a partfs translator, to which one gives a disk image, and which exposes the partitions of the disk image, using parted, and the parted-based storeio (settrans -c foos1 /hurd/storeio -T typed part:1:file:/home/samy/tmp/foo). This would be libnetfs-based.
*** Move the mount/umount logic from utils/{,u}mount.c into glibc.
*** Fix /proc/self. Look at [glibc]/hurd/lookup-retry.c for how FS RETRY MAGICAL lookups work.
*** Add a tool to trace system calls, by using gnumach's Syscall-Emulation, see http://www.gnu.org/software/hurd/gnumach-doc/Syscall-Emulation.html
*** Improve our FUSE library.
*** Add a relatime or lazytime option to ext2fs.
*** Fix our symlink translator.
*** Fix chmod on fifos: mkfifo foo ; sudo chmod g+w foo
*** Strengthen httpfs: it should append '/' to URL automatically, it should not fallback index.html itself, etc.
probably a lot more small easy issues.
*** Fix O_NOATIME, see https://buildd.debian.org/status/fetch.php?pkg=borgbackup&arch=hurd-i386&ver=1.0.2-1&stamp=1460838100
* The hurd in qemu
** Installing

Add yourself to the kvm group.  This group lets you start a kernel-ized virtual machine.
You can check to which groups you belong with:
#+BEGIN_SRC sh :results output :exports both
groups joshua
#+END_SRC

#+RESULTS:
:  http video audio

You can add yourself to the kvm group with:
#+BEGIN_SRC sh :results output :exports both
# gpasswd -a user kvm
#+END_SRC

*** manually installing

Don't do this unless you know what you are doing.   This method is really HARD.

#+BEGIN_SRC sh :results output
  $ wget https://people.debian.org/~sthibault/hurd-i386/installer/cdimage/netinst.iso
  $ qemu-img create hd0.img 4G
  $ qemu-system-i386 -net nic,model=ne2k_pci -net user               \
  -m 2G --no-reboot -machine kernel_irqchip=off -cdrom debian-sid-20160824-NETINST-1.iso \
 -enable-kvm -curses -drive cache=writeback,file=hd020160824.img,format=raw
#+END_SRC

If you do not specify format=raw, then you won't be able to write changes to your image.
If you do not specify -curses, then you will run the installer graphically, and it may not let you use the up or down arrow keys.
BUT using -curses, it may fail to install correctly...It will fail to install if you specify Dvorak as your keyboard, because the Hurd does not support dvorak.

Make sure you DO not put your user's home directory in a separate partition that is mounted on /home.  The current debian port does not support this.
*** using the pre-installed image Recommended

#+BEGIN_SRC sh :results output :exports both
$ wget http://people.debian.org/~sthibault/hurd-i386/debian-hurd.img.tar.gz
$ tar -xz < debian-hurd.img.tar.gz
$ kvm -m 1G -drive cache=writeback,file=$(echo debian-hurd-*.img)
#+END_SRC

Then resize the image

#+BEGIN_SRC sh :results output :exports both :dir ~/programming/gnu/hurd/vm/
qemu-img info debian-hurd-20160824.img
#+END_SRC

#+RESULTS:
: image: debian-hurd-20160824.img
: file format: raw
: virtual size: 2G (11736711168 bytes)
: disk size: 1.2G

Then you can run ~qemu-img resize file size~ to change the img size
#+BEGIN_SRC sh :results output :exports both
qemu-img resize debian-hurd-20160824.img +5G
#+END_SRC

Then you need to resize the partitions in the disk image after you have booted into it.

**** Things to do after installing the hurd
***** add a new user
~useradd username~
***** set up sudo
It would be nice to get your normal user to be able to install packages.  To do that:

#+BEGIN_SRC sh :results output :exports both
adduser joshua sudo
#+END_SRC

Alternatively, do it the long way.
#+BEGIN_SRC sh :results output :export code
$ apt-get install sudo
$ export EDITOR=nano
# visudo
#+END_SRC

#+RESULTS:

You'll want to add yourself the ability to run sudo, so add this line
~joshua ALL=(ALL) ALL~

Control-X will let you close the file.  Press "y" to save the file.
***** Enable automatic filesystem fscking of the / partition in

/etc/default/rcS

***** setting up your instant development environment
#+BEGIN_SRC sh :results output :exports both
apt-get update
apt-get install -y git mercurial emacs vim
apt-get build-dep -y hurd gnumach
git clone git://git.sv.gnu.org/hurd/hurd.git
git clone git://git.sv.gnu.org/hurd/gnumach.git
git clone git://git.sv.gnu.org/hurd/incubator.git
#+END_SRC
***** [[connecting to the hurd via ssh]]
This means that you can run the Hurd in qemu, and if it is super slow, I can develop for the Hurd in
GNU/Linux!

** Running
*** regularly running
#+BEGIN_SRC sh :results output :exports both
qemu-system-i386 -m 2G -net nic -net user,hostfwd=tcp::5555-:22 \
-drive file=hd0.img,cache=writeback,format=raw                  \
-curses -enable-kvm                                             \
-no-reboot
#+END_SRC

The ~-curses~ option runs the Hurd in a text layout.  It keeps your current keyboard layout.  Since I am using Dvorak, I need to pass this option.  The Hurd does not yet support dvorak.

The ~-enable-kvm~ option tries to run the Hurd as fast as possible by letting the virtual machine run as fast as possible on the CPU.

The ~-no-reboot~ option I found info [[https://www.gnu.org/software/hurd/contributing.html#index4h2][here]].

For shutting down, use reboot, then press c in grub and issue halt (to avoid filesystem corruption). Adding --no-reboot to the qemu line should help, too.

When you see graphic mode, press enter a couple of times
*** connecting to the hurd via ssh
It is often convenient to connect through ssh to the box, instead of logging on
the console, this can be done by forwarding the ssh port:

$ kvm -net user,hostfwd=tcp:127.0.0.1:2222-:22 -net nic -drive file=debian-hurd*.img,cache=writeback -m 1G

and then you can connect through ssh:

ssh user@localhost -p 2222

In Emacs you can do this with M-x find-file RET /METHOD:USER@HOST#PORT:FILENAME

aka /ssh:joshua@localhost#2222

*if you ssh into a new qemu install of the Hurd*, the old ssh won't work.  You'll have to remove the localhost line in

#+BEGIN_SRC sh :results output :exports both :dir ~/
cat .ssh/known_hosts
#+END_SRC

#+RESULTS:
: ssh.phx.nearlyfreespeech.net,208.94.116.211 ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEkD1YQOPCy0w/g/OqpBys337RAtyK/HDmW2i0BUW4mnN5em2jkB69nhJP91nU8s05OfB1MgxVVzyXc80Ounbzw=[localhost]:2222 ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN08IEOxaW//sIczLWdLxX2tqD5XCw6WyGapkRKXnx46by/MNADwrz2orWELkmyS6zz03YfeIrtEFQ7TyxF3IFw=

** Transferring files from host to qemu
Mounting Disk Image on Host

You may wish to mount your disk image on your host system to transfer files. To do this you will first need to find the offset of the partition you wish to mount.

#+BEGIN_SRC sh :results output :dir ~/programming/gnu/hurd/vm
 fdisk -l hd0.img
#+END_SRC

#+RESULTS:
#+begin_example
Disk hd0.img: 4 GiB, 4294967296 bytes, 8388608 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xb5b4d7a0

Device     Boot   Start     End Sectors  Size Id Type
hd0.img1           2048 7938047 7936000  3.8G 83 Linux
hd0.img2        7940094 8386559  446466  218M  5 Extended
hd0.img5        7940096 8386559  446464  218M 82 Linux swap / Solaris
#+end_example

Now take the number of sectors for the beginning of the partition and multiply it by the sector size. My / partition starts at sector 2048 and I have a sector size of 512 therefore my offset is 1048576.

The partition starts at 1955840.  What's my sector size?

#+BEGIN_SRC emacs-lisp
(* 1955840 512)
#+END_SRC

#+RESULTS:
: 1001390080

# mount -o loop,offset=1048576 hd0.img /mnt/diskimage

Then you mount the device on a directory! Not a file.

The actual command that I use is:
#+BEGIN_SRC sh :results output :exports both :dir /home/joshua/programming/gnu/hurd/vm
sudo mount -o loop,offset=1001390080 hd020160824.img ./web-page
#+END_SRC
** trouble booting the Hurd

youpi:  depending where it hangs, perhaps disabling the hur dconsole would
        help, by setting ENABLE to false in /etc/default/hurd-console

How to start the Hurd console from the mach console (info [[https://www.debian.org/ports/hurd/hurd-install.html][here]]):

~# console -d vga -d pc_mouse --repeat=mouse -d pc_kbd --repeat=kbd -d generic_speaker -c /dev/vcs~

** updating (on debian systems)
 sudo apt-get update        # Fetches the list of available updates
 sudo apt-get upgrade       # Strictly upgrades the current packages
 sudo apt-get dist-upgrade  # Installs updates (new ones)

 apt-get autoremove         #removes old packages
** hurd qemu notes
     This lets you write stuff to disk
     qemu-system-i386 -drive file=debian-hurd-20150320.img,cache=writeback,format=raw

     lets you start it in text mode AND lets you
     qemu-system-i386 -drive file=debian-hurd-20150320.img,cache=writeback,format=raw --curses

    interesting other options
    qemu-system-i386 -drive file=debian-hurd-20150320.img,cache=writeback,format=raw --curses -cpu core2duo -smp 2 -m 1G -enable-kvm --no-reboot

    it's a core2duo processor
    treat this thing like 2 cpus
    memory 1 gigabyte
    enable kvm, which will make it SOOO much faster
    don't reboot the system.  This avoids filesystem corruption

    start in full screen
    -full-screen

    qemu-system-i386 -drive file=debian-hurd-20150320.img,cache=writeback,format=raw --curses -cpu core2duo -smp 2 -m 1G -enable-kvm -full-screen

    --kvm info on arch wiki
    https://wiki.archlinux.org/index.php/QEMU#Enabling_KVM
    https://wiki.archlinux.org/index.php/KVM

    Checking support for KVM

Hardware support
KVM requires that the virtual machine host's processor has virtualization support (named VT-x for Intel processors and AMD-V for AMD processors). You can check whether your processor supports hardware virtualization with the following command:
$ lscpu
Your processor supports virtualization only if there is a line telling you so.
You can also run:
$ egrep --color=auto 'vmx|svm|0xc0f' /proc/cpuinfo
If nothing is displayed after running that command, then your processor does not support hardware virtualization, and you will not be able to use KVM.
Note: You may need to enable virtualization support in your BIOS.
Kernel support
Arch Linux kernels provide the appropriate kernel modules to support KVM and VIRTIO.
KVM modules
You can check if necessary modules (kvm and one of kvm_amd, kvm_intel) are available in your kernel with the following command (assuming your kernel is built with CONFIG_IKCONFIG_PROC):
$ zgrep CONFIG_KVM /proc/config.gz
If the module is not set equal to y or m, then the module is not available.
Para-virtualized devices

Para-virtualization provides a fast and efficient means of communication for guests to use devices on the host machine. KVM provides para-virtualized devices to virtual machines using the Virtio API as a layer between the hypervisor and guest.
All virtio devices have two parts: the host device and the guest driver.
VIRTIO modules
Use the following command to check if needed modules are available:
$ zgrep CONFIG_VIRTIO /proc/config.gz
Loading kernel modules
First, check if the kernel modules are automatically loaded. This should be the case with recent versions of udev.
$ lsmod | grep kvm
$ lsmod | grep virtio
In case the above commands return nothing, you need to load kernel modules.
Tip: If modprobing kvm_intel or kvm_amd fails but modprobing kvm succeeds, (and lscpu claims that hardware acceleration is supported), check your BIOS settings. Some vendors (especially laptop vendors) disable these processor extensions by default. To determine whether there's no hardware support or there is but the extensions are disabled in BIOS, the output from dmesg after having failed to modprobe will tell.
List of para-virtualized devices
network device (virtio-net)
block device (virtio-blk)
controller device (virtio-scsi)
serial device (virtio-serial)
balloon device (virtio-balloon)
How to use KVM

Enabling KVM
KVM must be supported by your processor and kernel, and necessary kernel modules must be loaded. See KVM for more information.
To start QEMU in KVM mode, append -enable-kvm to the additional start options. To check if KVM is enabled for a running VM, enter the QEMU Monitor using Ctrl+Alt+Shift+2, and type info kvm.
Note:
If you start your VM with a GUI tool and experience very bad performance, you should check for proper KVM support, as QEMU may be falling back to software emulation.
KVM needs to be enabled in order to start Windows 7 and Windows 8 properly without a blue screen.

I have to start qemu without curses in order to test if kvm has been enabled.  I also probably need to check my bios setting. Perhaps
my bios settings disable proper kvm support

I can add a new user with `adduser`  joshua babbages

I can use msr-tools to write to my cpu to enable intel virtual technology
rdmsr – read MSR from any CPU or all CPUs
wrmsr – write values to MSR on any CPU or all CPUs  THIS IS VERY DANGEROUS. directly writing to a cPU register could brick my system

cpuid – show identification and feature information of any CPU
https://askubuntu.com/questions/103965/how-to-determine-if-cpu-vt-extensions-enabled-in-bios/103966#103966
http://linux.koolsolutions.com/2009/09/19/howto-using-cpu-msr-tools-rdmsrwrmsr-in-debian-linux/

mach comes with its own kernel debugger
https://www.gnu.org/software/hurd/gnumach-doc/Kernel-Debugger.html
https://www.gnu.org/software/hurd/microkernel/mach/gnumach/debugging.html

** trouble shooting
**** Fixing corrupted filesystems

If you get logged into a root shell and your root filesystem is corrupted to fix it, is quite easy:

Use ~cat /etc/fstab~ to find the root filesystem, then run ~fsck.etx2 RootFileSystem~
Then run ~exit~ to shutdown.

If you are not able to access the root shell, then just run the command ~fix-hurd~, that I've created [[file:~/programming/bash/fix-hurd][here]].

     Also, it tells you that you will cause severe filesystem damage if you try to run the next command
     sh-2.05# fsysopts / -w
     sh-2.05# e2fsck /dev/hd0s1 <ENTER>  press enter multiple times
**** Running fsck when you can't boot the machine
:PROPERTIES:
:ID:       34ec4c89-0585-4e5f-8103-769426d5440e
:END:
use ~loseupt -P~  -P forces the kernel to scan the partition table on a newly create loop device

The offset for partition one is 512 * 2048 = 1048576

~losetup -o 1048576 --find --show hd0.img~

The above command will print out the loop device that the partition in installed on.  You can then run

~fsck.ext2 /dev/loop0~

These links seem to be some good information:

http://serverfault.com/questions/380186/how-to-run-fsck-on-guest-vms-from-kvm

http://www.ingent.net/en/tag/kvm/

[[gnus:nnimap+imap.fastmail.com:bug-hurd#1523201226.5641.7.camel@gmail.com][here is an email explaining this as well.]]

**** cannot update errors
***** Insert CD errors
add ~-cdrom SOMETHINGDEBIAN.iso~ to the qemu command
**** cannot update No public key KEY

If ~apt-get update~ fails, then try the following command:
#+BEGIN_SRC sh :results output :exports both
apt-get install debian-keyring
gpg --keyserver pgp.mit.edu --recv-keys 1F41B907
gpg --armor --export 1F41B907 | apt-key add -
#+END_SRC

KEY is a BIG long string.  In this case I've made it to be 1F41B907, but the string can be longer than that.
**** Hurd refuses to boot properly
#+BEGIN_SRC sh :results output :exports both
  ide0: reset: success
  hd0: irq timeout: status=0x50 { DriveReady SeekComplete }
  hd0: irq timeout: status=0x50 { DriveReady SeekComplete }
  hd0: irq timeout: status=0x50 { DriveReady SeekComplete }
  hd0: irq timeout: status=0x58 { DriveReady SeekComplete DataRequest }
  ide0: reset: success
  hd0: irq timeout: status=0x50 { DriveReady SeekComplete }
  end_request: I/O error, dev 03:01, sector 1100704
  hd0: irq timeout: status=0x50 { DriveReady SeekComplete }
  hd0: irq timeout: status=0x58 { DriveReady SeekComplete DataRequest }
  ide0: reset: success
  hd0: irq timeout: status=0x50 { DriveReady SeekComplete }
  hd0: irq timeout: status=0x50 { DriveReady SeekComplete }
  hd0: irq timeout: status=0x50 { DriveReady SeekComplete }
  hd0: irq timeout: status=0x58 { DriveReady SeekComplete DataRequest }
  ide0: reset: success
  hd0: status error: status=0x58 { DriveReady SeekComplete DataRequest }
  end_request: I/O error, dev 03:01, sector 5255176
  hd0: drive not ready for command
  hd0: irq timeout: status=0x50 { DriveReady SeekComplete }
#+END_SRC

Try disabling Hurd console:

 /etc/default/hurd-console

 You can still start it manually

How to start the Hurd console from the mach console (info [[https://www.debian.org/ports/hurd/hurd-install.html][here]]):

~# console -d vga -d pc_mouse --repeat=mouse -d pc_kbd --repeat=kbd -d generic_speaker -c /dev/vcs~
**** internet connectivity
***** You can reset the internet here: https://www.gnu.org/software/hurd/hurd/translator/pfinet.html

To find your gateway

#+BEGIN_SRC sh :results output :exports both
ip r
#+END_SRC

#+RESULTS:
: default via 172.16.112.1 dev enp0s9 proto static metric 100
: 172.16.112.0/22 dev enp0s9 proto kernel scope link src 172.16.113.155 metric 100

172.16.112.1 is the gateway.

#+BEGIN_SRC sh :results output :exports both

#+END_SRC
***** some qemu info

Network options

       -nic [tap|bridge|user|l2tpv3|vde|netmap|vhost-user|socket][,...][,mac=macaddr][,model=mn]
           This option is a shortcut for configuring both the on-board (default) guest NIC hardware and the host network backend in one go. The host
           backend options are the same as with the corresponding -netdev options below.  The guest NIC model can be set with model=modelname.  Use
           model=help to list the available device types.  The hardware MAC address can be set with mac=macaddr.

           The following two example do exactly the same, to show how -nic can be used to shorten the command line length (note that the e1000 is the
           default on i386, so the model=e1000 parameter could even be omitted here, too):

                   qemu-system-i386 -netdev user,id=n1,ipv6=off -device e1000,netdev=n1,mac=52:54:98:76:54:32
                   qemu-system-i386 -nic user,ipv6=off,model=e1000,mac=52:54:98:76:54:32

       -nic none
           Indicate that no network devices should be configured. It is used to override the default configuration (default NIC with "user" host network
           backend) which is activated if no other networking options are provided.

       -netdev user,id=id[,option][,option][,...]
           Configure user mode host network backend which requires no administrator privilege to run. Valid options are:

           id=id
               Assign symbolic name for use in monitor commands.

           ipv4=on|off and ipv6=on|off

** braunr apparently made vms use hugetlbfs for physical memory management?

This is good?

https://www.gnu.org/software/hurd/public_hurd_boxen/sceen.html
** qemu stuff for vms
http://lifeisabug.com/kvm-virtualization-arch-linux-host-system-qemu-virtio-hugepages-systemd/
* booting the Hurd from grub.  IE: grub normally boots Linux
#+BEGIN_SRC sh :results output :exports both
grub> multiboot /boot/gnumach.gz root=device:hd0s6
grub> module  /hurd/ext2fs.static ext2fs --readonly \
                   --multiboot-command-line='${kernel-command-line}' \
                   --host-priv-port='${host-port}' \
                   --device-master-port='${device-port}' \
                   --exec-server-task='${exec-task}' -T typed '${root}' \
                   '$(task-create)' '$(task-resume)'
grub> module /lib/ld.so.1 exec /hurd/exec '$(exec-task=task-create)'
grub> boot
#+END_SRC

I seem unable to boot the Hurd.  My computer's bios doesn't like it.  If my bios doesn't like it, I don't think that grub can boot from it
.
* building Hurd gnumach and MIG
** hurd/compiling Hurd
https://www.gnu.org/software/hurd/hurd/building.html

 building the Hurd: apt-get build-dep hurd
hurd-2017-02-28:21:25:56< teythoon> and the same for gnumach if you want to build that
hurd-2017-02-28-21:26:01< teythoon> then clone the repository
hurd-2017-02-28-21:26:08< teythoon> autoreconf -fi
hurd-2017-02-28-21:26:22< teythoon> the usual ./configure, make dance
hurd-2017-02-28-21:26:46< teythoon> building it shouldn't be too problematic
hurd-2017-02-28-21:27:02< teythoon> building something that boots is a bit more tricky
hurd-2017-02-28-21:27:08< teythoon> good luck ;)
** gnumach
https://www.gnu.org/software/hurd/microkernel/mach/gnumach/building.html

 building the Hurd: apt-get build-dep hurd
hurd-2017-02-28:21:25:56< teythoon> and the same for gnumach if you want to build that
hurd-2017-02-28-21:26:01< teythoon> then clone the repository
hurd-2017-02-28-21:26:08< teythoon> autoreconf -fi
hurd-2017-02-28-21:26:22< teythoon> the usual ./configure, make dance
hurd-2017-02-28-21:26:46< teythoon> building it shouldn't be too problematic
hurd-2017-02-28-21:27:02< teythoon> building something that boots is a bit more tricky
hurd-2017-02-28-21:27:08< teythoon> good luck ;)
** mig
https://www.gnu.org/software/hurd/microkernel/mach/mig/gnu_mig/building.html
* developing notes
    regular meeting take place every thursday at 19:00 UTC
     which is 3pm my time :)
     http://www.worldtimebuddy.com/utc-to-est-converter
     I can chat with hurd developers! woo hoo!

Make clean will remove previous builds and possible rebuild everything...
I'll see the errors again.
** hurd dev notes

pflocal is for /server/socket/1
pfinet runs on /server/socket/2 and is for networking
ipv6 runs on /server/socker/26

hostpriv means gnumach usually
a function that starts with vm_ will normally be found with mach, because mach manages memory
the proc server knows about processes and their UIDs.  This is needed because gnumach does not understand UIDs.  It doesn't understand
which process is root or a user
UID 0 is root
PROC is a macro that says this is proc
EPERM macro means not authorized or not root.
proc is started at bootstrap.  It allows processes root if they have UID 0
this lets a user to create a subhurd.
libshouldbeinlibc is the library where unstable functions that should be in libc should be kept.

** assign copyright to the FSF
Please email the following information to assign@gnu.org, and we
will send you the assignment form for your past and future changes.

Please use your full legal name (in ASCII characters) as the subject
line of the message.
----------------------------------------------------------------------
REQUEST: SEND FORM FOR PAST AND FUTURE CHANGES

[What is the name of the program or package you're contributing to?]

[Did you copy any files or text written by someone else in these changes?
Even if that material is free software, we need to know about it.]

[Do you have an employer who might have a basis to claim to own
your changes?  Do you attend a school which might make such a claim?]

[For the copyright registration, what country are you a citizen of?]

[What year were you born?]

[Please write your email address here.]

[Please write your postal address here.]

[Which files have you changed so far, and which new files have you written
so far?]

** mach

*** mach IPC
good for RPC, OO distributed programming, streaming of data, and sending lots of data.

*** boot with a modified mach
just apt build-dep gnumach
hurd-2017-06-26:14:36:23< teythoon> then clone gnumach
hurd-2017-06-26-14:37:20< teythoon> then configure it like this: ../configure --disable-wireless-group --disable-scsi-group --disable-floppy --disable-pcmcia-group --enable-kdb --enable-net-group
hurd-2017-06-26:14:37:36< teythoon> make, copy gnumach to /boot, update-grub, and reboot

** I can compile the texinfo source with texi2pdf <filename>.texi
** how to create a git diff
git diff <old commit hash> <newer commit hash> > patch.diff
Note that there can be many commits between the two hashes.  ie: one commit is from 2 months ago, and the second
commit is from today
** how to write a change log
https://www.gnu.org/software/hurd/source_repositories.html
https://www.gnu.org/prep/standards/html_node/Change-Logs.html
* roadmap
** hurd roadmap
*** fix libdiskfs locking issues

These days, the ext2fs server seems to cause most of the crashes on the Hurd.  It is believed that libdiskfs causes most of these issues.  A unit testing framework that checks locking in various code paths or a system that constantly checks locking correcting at runtime could help solve this issue.

This is a fairly involved project that must optimize locking issues in a multithreaded application.

https://www.gnu.org/software/hurd/community/gsoc/project_ideas/libdiskfs_locking.html
*** improve I/O performance

One obvious reason that the Hurd feels slow when compared to GNU/Linux is a slow I/O system performance, specifically very slow hard disk access.  The first task is to implement read-ahead, which tries to read more information from disk than in needed presently.  Then the excess data can be retrieved from RAM.  Even a simple read-ahead would be a huge performance improvement.

Other related tasks include scheduling reads and writes to minimize head movement, effective block caching, reading and writing multiple blocks at once, effective reads/writes to partial blocks, and various ext2 filesystem server optimizations.

https://www.gnu.org/software/hurd/community/gsoc/project_ideas/disk_io_performance.html

Read-ahead is an involved project, though not a research project.

*** porting valgrind to the hurd
Valgrind is an important debugging tool that checks for memory errors.  It can help discover existing bugs in the hurd servers as well as aid in porting programs to the Hurd.

There is an unfinished start of port to the Hurd, and it is not an easy task.  However, it is a fairly predictable one.  One must learn how valgrind handles system calls and the Hurd RPCs.  No major design or research is necessary.

https://www.gnu.org/software/hurd/community/gsoc/project_ideas/valgrind.html
*** new driver framework
The Hurd currently uses some hardware drivers in the mach kernel.  These drivers are old, which potentially makes the Hurd have a hard time supporting newer machines. The most promising solution is to implement drivers from the rump kernel, which uses netbsd code.

This is a fairly involved project that will probably require previous device driver experience.

https://www.gnu.org/software/hurd/community/gsoc/project_ideas/driver_glue_code.html
*** kernel instrumentation
The Hurd suffers from poor performance, and while its developers know many of the problems that cause poor performance, they need better tools to diagnosis performance bottlenecks. The Hurd has some existing kernel probes, but not all of the patches were functional and only some were accepted into the project.  The most reasonable solution is to improve the existing kernel probes and a nice frontend for writing scripts to control the probes.

This is an involved project that requires knowledge of low level programming.  It requires a good knowledge of gnumach and interactions of complex systems.

https://www.gnu.org/software/hurd/community/gsoc/project_ideas/dtrace.html
*** file locking support
UNIX has several different file locking mechanisms.  Some of them work on the hurd while others are buggy or non-existent.  This breaks numerous applications.  Ideally all file locking mechanisms should work properly.  This is not a trivial task and implementing or correcting existing file locking will require a competent developer.

https://www.gnu.org/software/hurd/community/gsoc/project_ideas/file_locking.html
*** NEWBIE IDEAS
**** improve glibc for the hurd and fix compatibility problems
https://www.gnu.org/software/hurd/community/gsoc/project_ideas/testsuites.html

The Hurd uses glibc to forward POSIX requests to various Hurd RPCs protocals.  The Hurd is missing several POSIX and other standards.  Some of these tasks can be handled by less experienced developers, and others will require immense work.  This might be a good area to begin Hurd development.
**** gnumach code clean up
https://www.gnu.org/software/hurd/community/gsoc/project_ideas/gnumach_cleanup.html

The Hurd runs on top of the gnumach microkernel, which is in a fairly bad shape.  Gnumach is a modified version of the research CMU Mach, which has bits of BSD code.  Parts of CMU Mach ripped out many UNIX mechanisms to be replaced by userspace processes.  The result is a code base that is hard to change, difficult to support modern hardware, or even to fix bugs.

One can begin by fixing all of the compiler warnings, restructuring code, removing dead or unused code paths, etc.  You should have a good knowledge of C and experience in working in large C code bases.  This is not necessarily a task that requires a lot of research, but it is probably a lot of busy work and careful changes to the existing code.
**** automated testing
https://www.gnu.org/software/hurd/community/gsoc/project_ideas/testing_framework.html

https://www.gnu.org/software/hurd/open_issues/unit_testing.html

Large complex software often employs a testing framework to reliably fix bugs and ensure that no bug appears twice.  Ideally, when a developer fixes a bug, he would write a test to ensure that the bug would not occur again.  This also applies when adding a new feature.  Since there is no testing framework, it's harder to ensure that the Hurd does not add a new bug with a new feature.

This project requires a knowledge of C build systems and unit frameworks, but one need not be a Hurd expert to complete this project.  This is an excellent project to get one started on Hurd development.

*** improve the GDB port for the hurd
https://www.gnu.org/software/hurd/community/gsoc/project_ideas/gdb.html

GDB is a helpful debugging tool for the Hurd, but the Hurd port has some bugs and lacks some features.  You can help by fixing bugs and implementing some of the features like gdbserver.  It would also be nice if GDB could properly debug a server that calls another server.  For example if you are debugging the ext2fs server, gdb should be able to follow code execution that uses libdiskfs.

You will need to learn how GDB works generally, how the Hurd port works, and some general knowledge of the Hurd and gnumach.
*** usb support/sound support

Many new laptops today lack a CD drive, which makes it harder to install the Hurd, and USB support is an important feature.  The recommended way to fix this is to run a rump kernel in a userspace process.  The rump kernel would support the usb hardware, and the Hurd would do the rest.

The Hurd has initial sound support, but it could be improved.  The initial patches work, but they are not the perfect solution.  An ideal solution allows changing the initial code to be more Hurdish.  You can improve the current implementation or fix any existing bugs.

This requires good knowledge of the Hurd and gnumach as well as experience programming in C.
*** Guix/Hurd
**** startup in scheme

Currently only Debian GNU/Hurd is the only distribution that runs the GNU Hurd.  While the Hurd works well on debian, many developers want to see a GNU GuixSD/Hurd.  GNU Guix is GNU's new package manager with many forward looking features.  While GNU Guix already runs on the Hurd, the Hurd's startup is still based on a Debian approach.  GNU GuixSD uses the init system Shepherd to boot up.  Shepherd needs to be ported over to work on the Hurd.

**** gnu package manager guix
https://www.gnu.org/software/hurd/community/gsoc/project_ideas/package_manager.html

Most package managers use non-trivial databases to help one easily install, delete, and update software.  The Hurd can use a much better solution: stowfs. Stowfs can create a virtual directory tree, on the fly, without using any global databases.  This can ease the development burden of maintaining guix on the Hurd.

This project includes improving GNU Guix, GNU's package manager to the Hurd.  It will require knowledge of gnu guile and C.  One will have to modify GNU Guix to use stowfs.
*** 64 bit

More processors are 64 bit processors, and users expect the benefits of 64 bit processors.  Unfortunately, the Hurd only runs 32 bit.  This is not a huge issue, because 64 bit machines still can run in 32 bit mode.  However, the Hurd should eventually be able to run in 64 bit mode.

Because of some deficiencies in gnumach, a 64 bit kernel space and a 64 bit user space is a little too difficult to implement.  An easier fix is to run the Hurd in a 64 bit kernel space and a 32 bit userspace process.  The kernel can already run in a 64 bit, but more work is needed for a 64 bit to 32 bit conversion.
*** make some stub libraries
https://www.gnu.org/software/hurd/community/gsoc/project_ideas/hardware_libs.html

Numerous programs link to various libraries to provide ranging hardware support.  Examples include libusb and libbluetooth.  The Hurd doesn't support these devices, but the programs could still be built if the Hurd had dummy libraries returning appropriate error codes (ie: no device found).  Many of these programs would still be useful without the ability to access usb or bluetooth.

The goal is to get a dummy library built for the Hurd and package it in Debian GNU/Hurd.  This requires learning a bit about the libraries and debian packaging, but it does not require intense research.
*** cd audio grabbing
https://www.gnu.org/software/hurd/community/gsoc/project_ideas/cdparanoia.html

The hurd supports CD-ROMs, but not for extracting audio files, which means that many applications will not build on the Hurd.  Adding support for audio extraction should not be too hard.  It will involve learning about the Hurd and ATAPI commands, but it is certainly doable.

*** improve perl/python support
https://www.gnu.org/software/hurd/community/gsoc/project_ideas/perl_python.html

The Hurd supports perl and python, but the implementation is not complete and both language's test suite report some failures.  One can work on fixing these test suite failures. Perl is more important and has more failures.  This will involve a fair amount of C programming and may mean diving deeply into the Hurd's internals.

*** DONE physical memory management buddy allocator
CLOSED: [2017-04-23 Sun 14:35]
:LOGBOOK:
- State "DONE"       from              [2017-04-23 Sun 14:35]
:END:
https://www.gnu.org/software/hurd/community/gsoc/project_ideas/physical_memory_management.html

*** DONE implement xattr support
CLOSED: [2017-04-23 Sun 14:35]
:LOGBOOK:
- State "DONE"       from              [2017-04-23 Sun 14:35]
:END:
https://www.gnu.org/software/hurd/community/gsoc/project_ideas/xattr.html

** gnumach roadmap
file:///home/joshua/programming/gnu/hurd/vm/web-page/home/joshua/programming/web.rendered/microkernel/mach/gnumach/projects.html

*** Improve Mach's IPC mechanism: Mach 5

The Mach IPC mechanism has some deficiencies, many of which cause mach's poor performance.

**** Type Descriptor rework

A mach message contains pairs for type descriptors and values.  Each type descriptors describes the kind of data and the amount of data.  The kernel needs to parse the message to rewrite rights and pointers.  The current format describes a type followed by a value, then another type followed by a value, etc.

The change to the format involves moving the type descriptors to the beginning of the message, which will create an index into the data.  One type of type descriptors is machmsgtypelongt, which is unneeded.  This type descriptors can be removed.

Luckily the Mach4 type descriptor contains one unused bit.  This bit can be used to indicate that the message is of type Mach 5, which would provide an easy transition to the new ABI.
*** Faster Syscall Interface
Currently gnumach uses trap gates to enter in the kernel on i386.  Another solution is to use sysenter, which is not available on all platforms.  Linux fixes this by using the VDSO mechanism, which we can do too.  We believe that implementing sysenter/sysexit, mach's IPC could have a 10% boost in performance.

There is an existing prototype.

https://lists.gnu.org/archive/html/bug-hurd/2015-05/msg00000.html
https://lists.gnu.org/archive/html/bug-hurd/2016-09/msg00056.html
*** Interface for userspace drivers
The rump kernel seems to be the best option for this.  There is a rump kernel prototype that has been implemented.  It just needs to be tweaked to be more hurdish.

https://lists.gnu.org/archive/html/bug-hurd/2016-02/msg00126.html

** gnu MIG roadmap
gnumach is an IDL compiler.  It takes a specification and generates code that simplifies the remote procedure calls.  It has some issues.

I can get more ideas from here:  file:///home/joshua/programming/gnu/hurd/vm/web-page/home/joshua/programming/web.rendered/tag/open_issue_mig.html

*** GNU MIG is not good at sending structured data
file:///home/joshua/programming/gnu/hurd/vm/web-page/home/joshua/programming/web.rendered/open_issues/mig_portable_rpc_declarations.html

Some servers want to be able to send lots of complex data.  Imagine a server that sends email using a remote smtp server.  To send the email, the hurd server needs your email, password, and connection information.  All this information ought to be sent in some type of data structure.  MIG is not suited for this, nor can it do this.

MIG ought to be modified to allow for handling arbitrarily complex data structures.  This may not be an easy task, and it might require a lot of research to find the best solution.

*** Duplicate inclusion guards
file:///home/joshua/programming/gnu/hurd/vm/web-page/home/joshua/programming/web.rendered/open_issues/duplicate_inclusion_guards.html

*** MIG does not call functions on when the reply is an error
file:///home/joshua/programming/gnu/hurd/vm/web-page/home/joshua/programming/web.rendered/open_issues/mig_error_reply.html
*** MIGs variable length strings support is broken
*** What is low level serialization?

MIG apparently tries to do serialization/parsing and message sending/receiving, which I guess should be separate?
file:///home/joshua/programming/gnu/hurd/vm/web-page/home/joshua/programming/web.rendered/microkernel/mach/mig/documentation.html

Braunr plans to do that in x15
* some hurd tasks
[[file:/ssh:joshua@localhost#2222:/home/joshua/programming/hurd/TODO::-*-%20Mode:%20Outline%20-*-][todo]]

[[file:/ssh:joshua@localhost#2222:/home/joshua/programming/hurd/tasks::GNU%20Hurd%20Task%20List%20Version%201.23.%20Last%20updated%2025%20April%202006.][tasks]]
** port packages to guix!
* some things I can do now
** hurd
*** get rid of p-thread to do task
[[file:/ssh:joshua@localhost#2222:/home/joshua/programming/hurd/tasks::*%20Get%20pthreads%20working%20and%20integrated.][we can get rid of this p-thread todo task]]
*** hurd.texi
**** [[file:/ssh:joshua@localhost#2222:/home/joshua/programming/hurd/doc/hurd.texi::and%20it%20requires%20superuser%20privileges%20to%20control%20the%20new%20Hurd%20(or%20allow][subhurd does not require superuser privileges]]
**** [[file:/ssh:joshua@localhost#2222:/home/joshua/programming/hurd/doc/hurd.texi::All%20Hurd%20servers%20and%20libraries%20are%20aggressively%20multithreaded%20in%20order][hurd does not use cthreads anymore]]
[[file:/ssh:joshua@localhost#2222:/home/joshua/programming/hurd/libthreads/cthreads.c::/*][but they still have it in the source code]]
**** [[file:/ssh:joshua@localhost#2222:/home/joshua/programming/hurd/doc/hurd.texi::filesystem%20and%20networking%20helper%20libraries.%20@code{libiohelp}%20requires][does i/o require libthreads]] ?
[[file:/ssh:joshua@localhost#2222:/home/joshua/programming/hurd/libthreads/cthreads.c::/*][but they still have it in the source code]]
*** learn argp in C, this will help me with using trivfs
*** learn some C
What's a struct?  What's a union?

A struct is a collection of things in one variable name.

A union is a collection of things in one variable name, BUT only one thing may have data in it at a time.
*** learn some gnu build system.  This will help me with porting
*** update the project ideas page
https://www.gnu.org/software/hurd/community/gsoc/project_ideas.html

More ideas are here:
https://www.mail-archive.com/debian-hurd@lists.debian.org/msg25179.html
** take MIT open courseware courses
https://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-s096-effective-programming-in-c-and-c-january-iap-2014/

https://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-171-software-engineering-for-web-applications-fall-2003/

https://ocw.mit.edu/courses/comparative-media-studies-writing/cms-611j-creating-video-games-fall-2014/

https://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-033-computer-system-engineering-spring-2009/

All the online text books!

https://ocw.mit.edu/courses/online-textbooks/#electrical-engineering-and-computer-science
** when the Hurd gets a new feature I can mention it the [[https://www.gnu.org/software/hurd/contributing/web_pages/news/qoth_next.html][qoth next page]]

This is the page where the Hurd talks about new features that it got.
** improve the Hurd users guide
https://www.gnu.org/software/hurd/users-guide/using_gnuhurd.html

I can delete the section dealing with CVS
* wiki
** wiki things that need changed
*** DONE fix this broken link the relevant email is [[gnus:nnimap+imap.dismail.de:Bug%20Hurd#4595617e-8d71-629b-f2a9-bb3584889ff9@treenet.co.nz][here]].
CLOSED: [2019-01-04 Fri 07:28]
:LOGBOOK:
- State "DONE"       from              [2019-01-04 Fri 07:28]
:END:
*** DONE fix the pfinet and lwip that mention -i eth0  -> -i /dev/eth0
CLOSED: [2019-01-07 Mon 14:16]
:LOGBOOK:
- State "DONE"       from              [2019-01-07 Mon 14:16]
:END:

file:///home/joshua/prog/gnu/hurd/hurd-web.rendered/hurd/translator/lwip.html
*** DONE add some info to what a firmlink is on [[file:~/programming/gnu/hurd/hurd-web/hurd/translator/firmlink.mdwn::#%20IRC,%20freenode,%20#hurd,%202012-07-20][this]] page.
CLOSED: [2019-01-07 Mon 16:24]
:LOGBOOK:
- State "DONE"       from              [2019-01-07 Mon 16:24]
:END:

#+BEGIN_SRC sh :results output :exports both
grep -C 20 -r firmlink | less
#+END_SRC

I can use this command to search through the irc logs to find out more about the info that I want to know about.
*** update the gnumach memory management issues pages
https://www.gnu.org/software/hurd/microkernel/mach/gnumach/memory_management.html

https://www.gnu.org/software/hurd/open_issues/gnumach_memory_management.html

https://www.gnu.org/software/hurd/open_issues/gnumach_memory_management_2.html
*** make sure the title on the website says GNU/Hurd not just GNU

[[gnus:Bug%20Hurd#20190325-151106.sv114059.84365@savannah.gnu.org][Email from Benjamin Cardinal: {bug #56000} Website postfix i]]
*** add in some documentation about why the Hurd has ext2 and not ext4

[[gnus:Debian%20Hurd#CAH8yC8mBjw=JjP54g3iMfybK=k222rafVZ_L+iUT9fi_jquGLQ@mail.gmail.com][Email from Jeffrey Walton: Re: Debian GNU/Hurd 2019 relea]]
*** add some info to these store pages:

- [[file:~/prog/gnu/hurd/hurd-web/hurd/libstore/concat_store.mdwn::%5B%5B!meta%20title="concat%20store:%20concatenate"%5D%5D][concat store page]]
- [[file:~/prog/gnu/hurd/hurd-web/hurd/libstore/query_store.mdwn::%5B%5B!meta%20title="query%20store"%5D%5D][query store]]
- [[file:~/prog/gnu/hurd/hurd-web/hurd/libstore/remap_store.mdwn::%5B%5B!meta%20license="""%5B%5B!toggle%20id="license"%20text="GFDL%201.2+"%5D%5D%5B%5B!toggleable][remap store]]
- [[file:~/prog/gnu/hurd/hurd-web/hurd/libstore/task_store.mdwn::%5B%5B!meta%20copyright="Copyright%20%C2%A9%202007,%202008,%202009%20Free%20Software%20Foundation,][task store]]
- etc
*** add some more details to the Hurd libraries pages
The [[file:~/prog/gnu/hurd/hurd-web/hurd/libnetfs.mdwn::%5B%5B!meta%20copyright="Copyright%20%C2%A9%202007,%202008,%202010%20Free%20Software%20Foundation,][libnetfs]] page is an awesome example of what all pages should be!

All the others could use some work!
 - libports
 - libstore
 - libchannel
 - libtrivfs
 - libdiskfs
 - libihash
 - libpthread
 - libfshelp
 - libps
*** is the tmpfs translator broken?  'cause [[file:~/prog/gnu/hurd/hurd-web/hurd/libstore/examples/ramdisk.mdwn::A%20(better)%20alternative%20would%20be%20using%20the%20%5B%5B`tmpfs`][this]] page says it is.
*** add some more detail to the hurd/gnumach debugging page on the hurd wiki
[[file:~/prog/gnu/hurd/hurd-web/open_issues/debugging.mdwn::*%20%5B%5BGNU%20Mach%20debugging|microkernel/mach/gnumach/debugging%5D%5D][general open issues debugging]]
**** [[file:~/prog/gnu/hurd/hurd-web/hurd/debugging.mdwn::#%20Strategies][hurd debugging]]
***** [[file:~/prog/gnu/hurd/hurd-web/hurd/debugging.mdwn::#%20Strategies][hurd rpctrace]]
Braunr doesn't really like rpctrace.  He'd rather use syscall tracing like strace.

I guess the gsync patches and rpctrace did not play well together.  So youpi had to do some magic to get them to play nicely.

rpctrace is basically a man-in-the-middle, and that'll have side effects.

Parallel tracing is completely asyncronous and out of the way.
**** [[file:~/prog/gnu/hurd/hurd-web/microkernel/mach/gnumach/debugging.mdwn::#%20Kernel%20Debugger%20(KDB)][debugging gnumach]]
*** read this [[file:~/prog/gnu/hurd/hurd-web/hurd/rpc.mdwn::%5B%5BRemote%20procedure%20call|/rpc%5D%5Ds%20are%20the%20basis%20for%20about%20everything%20in%20the%20Hurd.][rpc page]].  It's got great detail!
*** I can mention that one can download the old gnuhurd emails, but should mention how to convert to maildir [2/3]

http://batleth.sapienti-sat.org/projects/mb2md/

mb2md -d .mail/bug-hurd-archive-md

run this command first

mb2md -m -d ~/.mail   will tell mb2md where to store the maildirs...

mb2md -s sourcefile -d destdir

take a mbox file and convert it to the destination area.

mb2md -s sourcedir  -f somefolder
  mb2md -s ~/.mail/bug-hurd-archive  -f ~/.mail/bug-hurd-archive-md

take a source directory of mboxs and convert it to maildirs

mb2md -s ~/.mail/bug-hurd-archive

produced

~/Maildir/{cur,tmp,new} and

~/Maildir/.{2018-0}{1,2,3,4,5,6,7} each one of these hidden files had many files underneith them.

It did not produce a ~/Maildir/bug-hurd-archive-md/  where each file is an email address.
**** DONE mentioned how to download the emails
CLOSED: [2018-12-13 Thu 11:09]
:LOGBOOK:
- State "DONE"       from              [2018-12-13 Thu 11:09]
:END:
**** DONE mentioned the tool to convert to maildir
CLOSED: [2018-12-13 Thu 11:43]
:LOGBOOK:
- State "DONE"       from              [2018-12-13 Thu 11:43]
:END:
**** TODO did not show how to convert to maildir
:LOGBOOK:
- State "TODO"       from              [2018-12-13 Thu 11:43]
:END:
*** how about a crazy hurd possibilities page?

I can add a page on the Hurd to talk about some of the crazy things that you could do with the Hurd!?

Suppose that you have a large source code.  You would like to find all occurancies of "(TODO|FIXME)"

You could do something like this:

#+BEGIN_SRC sh :results output :exports both
settrans -pa /hurd/run  grep -r '(TODO|FIXME)' TODOs
#+END_SRC

Now every time that you read the TODOs, it'll grep -r through the whole directory, and point out all of your TODOs and FIXMEs!  You never have to run that command again.  You can just browse the file.
*** add a more concrete example of the confused deputy problem on the capability page
file:///home/joshua/prog/gnu/hurd/hurd-web.rendered/capability.html
*** I can make the mobile navigation menu vertical like a bootstrap menu
[[file:~/prog/gnu/hurd/hurd-web/.templates/page.tmpl::<TMPL_IF%20HTML5><TMPL_ELSE>
</TMPL_IF>][page.tmpl]]  This is the main page template.

I should also add in CSS on the [[file:~/prog/gnu/hurd/hurd-web/local.css::/*%20ikiwiki%20local%20style%20sheet][local.css]] file.
*** I can get some info from this task list
file:///home/joshua/programming/gnu/hurd/hurd-web.rendered/open_issues/some_todo_list.html
*** add an open issues PyHurd page.
*** add a tutorial showing someone how to run the python, perl, gcc, gdb, glibc, etc. testsuite failures.
*** add a page showing off the status of the language bindings:

- perl
- java
- python
- lisp
*** sort all of the files in the unsorted directory
[[file:~/programming/gnu/hurd/hurd-web.rendered/unsorted/][file:~/programming/gnu/hurd/hurd-web.rendered/unsorted/]]
*** fix this [[file:~/programming/gnu/hurd/hurd-web/documentation.mdwn::text="%5Bshapiro\_capintro\_1999%5D"%5D%5D.][broken link]] on the wiki. It goes no where.
:PROPERTIES:
:ID:       07792db9-7f0f-44ba-a166-45fd9c2df335
:END:
*** I should add a google summer of code project to make the mach message simplier and more modern

I sent an email asking for suggestions. As well as writing some down.

*** namespace translators I should document that you can (fopen ".,,0/") opens a directory tree with translators disabled!

Or you could mount a translator to unzip a file via "files.zip/gunzip"!

[[file:~/prog/gnu/hurd/hurd-web/community/gsoc/project_ideas/namespace-based_translator_selection.mdwn::A%20possible%20solution%20is%20implementing%20a%20mechanism%20for%20selecting%20translators][hurdish namespace based translators!]]

[[file:~/prog/gnu/hurd/hurd-web/community/gsoc/project_ideas/namespace-based_translator_selection/discussion.mdwn::%5B%5B!meta%20copyright="Copyright%20%C2%A9%202012%20Free%20Software%20Foundation,%20Inc."%5D%5D][discussion about it]]

file:///home/joshua/prog/gnu/hurd/hurd-web.rendered/hurd/translator/nsmux.html

apparently the Hurd has a way of opening up a special file, which auto starts a translator!

I should show off some of the things that it can do.
*** put some of Neal's paper's on the wiki

http://walfield.org/pub/people/neal/papers/hurd-misc/
*** add to the list of tiny project ideas

We could make rm -rf work faster
file:///home/joshua/prog/gnu/hurd/hurd-web.rendered/open_issues/rm_fr.html
*** create a list of ideas that would be suitable for a thesis!

A new rpctrace
new filesystems
etc.
*** I can apparently change mach's console keymap?  That's kind of cool!
Perhaps I should add an updated entry about this.

open_issues/keymap_mach_console.html
*** copy/read the arch hurd installation page somewhere.  it's got lots of good details.
The debian wiki should be updated, or this information should be stored on the hurd wiki somewhere.
https://wiki.archhurd.org/index.php/Installation_Guide#A_note_on_hardware_support
*** merge the faq with the old faq and older hurd faq

file:///home/joshua/programming/gnu/hurd/hurd-web.rendered/faq.html

file:///home/joshua/programming/gnu/hurd/hurd-web.rendered/faq/old_hurd_faq.html

file:///home/joshua/programming/gnu/hurd/hurd-web.rendered/faq/old_faq.html

Apparently [[file:~/programming/gnu/hurd/hurd-web/faq/old_faq.txt][defragging my filesystem]] will destroy GNU/Hurd translators.  Is this still true?

*** write some introductory material for the anatomy of a hurd system

file:///home/joshua/programming/gnu/hurd/hurd-web.rendered/open_issues/anatomy_of_a_hurd_system.html

https://teythoon.cryptobitch.de/posts/bootstrapping-the-hurd/

GNU/Hurd startup process is [[file:~/programming/gnu/hurd/hurd/daemons/runsystem.sh::#!/bin/bash][here.]]
*** remove items from the open issue page.

systemd
*** explain the Hurd Boot process
*** add some info to this page
https://www.gnu.org/software/hurd/microkernel/mach/gnumach/interface/syscall.html

It's currently very empty

I did add a mach_msg segment.
*** test that the symlink is still not working
file:///home/joshua/programming/gnu/hurd/hurd-web.rendered/open_issues/symlink_translator.html

Hmm.  It does still seem to be broken.
*** add in some Hurd possibilities
 use this
https://archive.fosdem.org/2018/schedule/event/microkernel_hurd_pci_arbiter/
talk to mention some of the hurd possibilities.

take a snapshot of the Hurd possibilities to explain the translators.
*** change the savannah repo to not point at an old repo

The savannah repo online points to an empty repository.  This should be changed.
*** remove implement xattr support from the open issues page
*** visudo works just fine, we can remove this webpage https://www.gnu.org/software/hurd/open_issues/visudo.html
*** the link on this page goes nowhere
file:///home/joshua/programming/gnu/hurd/hurd-web.rendered/open_issues/xorg_porting.html
*** delete the wine page?
*** secure chroot implementation is done?

http://www.gnu.org/software/hurd/community/gsoc/project_ideas.html
*** add my youtube channel to the Hurd

file:///home/joshua/programming/gnu/hurd/hurd-web.rendered/community/facebook.html

They mention the facebook channel!
*** do we need gnumig to compile libc?

file:///home/joshua/programming/gnu/hurd/hurd-web.rendered/microkernel/mach/mig/gnu_mig.html
*** add arch specific commands to start, install, and run the hurd in a VM
*** there's a webpage that mentions a bridging translator would be cool, but we already have one?

https://www.gnu.org/software/hurd/hurd/running/qemu.html#index8h1

https://www.gnu.org/software/hurd/hurd/translator/wishlist.html

Search for bridg  in both of these pages.  They contradict?
*** copy this page info into the  mach 5 proposal

file:///home/joshua/prog/gnu/hurd/hurd-web.rendered/open_issues/rework_gnumach_ipc_spaces.html

file:///home/joshua/prog/gnu/hurd/hurd-web.rendered/microkernel/mach/gnumach/projects/mach_5.html
** add a wiki page about what to do after running the hurd for the first time
** DONE writing a 2018 Q4 qoth
CLOSED: [2019-01-04 Fri 07:26]
:LOGBOOK:
- State "DONE"       from              [2019-01-04 Fri 07:26]
:END:
file:///home/joshua/programming/gnu/hurd/hurd-web.rendered/community/weblogs/ArneBab/how-i-write-a-qoth.html

file:///home/joshua/programming/gnu/hurd/hurd-web.rendered/contributing/web_pages/news/qoth_next.html

http://lists.gnu.org/archive/html/bug-hurd/YYYY-MM/threads.html
http://lists.debian.org/debian-hurd/YYYY/MM/
*** pci work
**** merged PCI_Arbiter and should be able to use on Debian GNU/Hurd
http://lists.gnu.org/archive/html/bug-hurd/2018-10/msg00029.html

futher polishing by Damien Zammit
http://lists.gnu.org/archive/html/bug-hurd/2018-11/msg00033.html
**** making libpciaccess support the hurd
AKA only one process can access the PCI config ports
https://lists.x.org/archives/xorg-devel/2018-November/057677.html
https://lists.x.org/archives/xorg-devel/2018-November/057691.html
*** Joan Lledo` updated lwip to work with the latest lwip
http://lists.gnu.org/archive/html/bug-hurd/2018-11/msg00062.html
*** Damien Zammit created an ACPI translator
http://lists.gnu.org/archive/html/bug-hurd/2018-11/msg00049.html
*** Samuel Thibault allows all users to make tmpfs
http://lists.gnu.org/archive/html/bug-hurd/2018-11/msg00012.html
*** Samuel fixed a bfs issue with /proc not being mounted
https://lists.debian.org/debian-hurd/2018/10/msg00007.html
*** LLVM now supports the Hurd
https://www.phoronix.com/scan.php?page=news_item&px=GNU-Hurd-LLVM-Clang
*** porting Rust and Go is ongoing
https://lists.debian.org/debian-hurd/2018/11/msg00020.html
*** add file record locking support  was this added?
http://lists.gnu.org/archive/html/bug-hurd/2018-11/msg00058.html
** setting up the wiki

 #+BEGIN_SRC sh :results output :exports both :dir ~/programming/gnu/hurd/web
 ikiwiki --setup ikiwiki.setup
 #+END_SRC

 #+RESULTS:

You should also be able to render the wiki locally via the script: render_locally
** Ikiwiki does not seem to be able to render the wiki locally easily...
* savannah task list
https://savannah.gnu.org/task/?group=hurd
** update the rewrite pfinet page
https://savannah.gnu.org/task/?5469
** is the firewall filter done?
https://savannah.gnu.org/task/?12723
** can we delete gopherfs task?
https://savannah.gnu.org/task/?10598
* a link to posix stuff http://pubs.opengroup.org/onlinepubs/9699919799/
* port programs to the Hurd.  Debian has several programs that ought to be packaged

https://www.gnu.org/software/hurd/community/gsoc/project_ideas/maxpath.html

Samuel's graph of packages that need porting.  They are grouped by the ones that are super important.

https://people.debian.org/~sthibault/graph-radial.pdf

There's a paragraph starting with "Pick One" that gives suggestions about how I can pick what packages to port.

There's also some info about porting on the Hurd site:  https://www.gnu.org/software/hurd/hurd/porting/guidelines.html

I should also check out the developer's corner.

And I should learn about the gnu build tools or Autoconf.  I should also probably learn about automake and libtool.

The gnu build system combines Autoconf, Automake, and Libtool.

gnulib is a C library of common code that should be shared among software packages.

libtool is gnu's solution to help developers make portable libraries.
* port programs for debian packaging:
debian-hurd@lists.debian.org
* lots of hurd translators use
This is a glibc function that stores a file in memory.  AKA you can read and write the entire file in RAM.  It's rather low level.

[[info:libc#Memory-mapped%20I/O][info:libc#Memory-mapped I/O]]
* lkl linux kernel library use Linux as a C library!

https://github.com/lkl
* what to do after you installed the Hurd
** Change the root password

gpasswd RET
Type in your new password RET
Type in your new password once more RET
** create a new user

adduser <username>
** install sudo
apt-get install sudo
** add your user to the sudo group
usermod -aG sudo <username>
** logout and login as your new user
** perform all updates

#+BEGIN_SRC sh :results output :exports both
sudo apt-get update        # Fetches the list of available updates
sudo apt-get upgrade       # Strictly upgrades the current packages
sudo apt-get dist-upgrade  # Installs updates (new ones)
#+END_SRC

You can learn more about apt-get via "man apt-get"

If apt warns you that it is missing a gpg key, then you can use apt-key to download said key.

See man apt-key for details.

I had an issue with a missing key.  I may have imported it with

apt-key adv --recv-key <key ID>

https://serverfault.com/questions/906972/the-following-signatures-couldnt-be-verified-because-the-public-key-is-not-avai
* managing job sets in bash
C-z suspends the current job.

jobs lists all current jobs

fg <number>  brings a job back to the foreground.

Jobs ARE NOT processes.

jobs are user processes mapped to lower numbers.

ie:  a find command can be mapped to job 2

The find command may have 1 or 2 processes running
* lwip Hurd blog
http://darnassus.sceen.net/~jlledom/en/index.html
* making a computer operating system
https://www.nand2tetris.org/
* writing a test to prove that I know the Hurd
** What is a translator?

:ANSWER:
A translator is a program attached to a filesystem node.  When the node is accessed, the translator runs and returns data instead of the filesystem node.  This can be used to transform the underlying data.
:END:
** What is port?
:ANSWER:
A port is a inter-process communication (IPC) channels implemented in the kernel.  They are similar to UNIX pipes.  Only one task can receive messages from a port.  Many tasks can send messages to the same port.

:END:

** What's a port bucket?
** What's a port class?
** What's a capability?
** What's a page?

:ANSWER:
In virtual memory, a page is region of memory.  Each page is a set width, something like 4KB or more.  Sometimes, you can have applications consuming 5GB of memory, but you only have 4GB of ram.  How does this work?

Well, an applications' page will get written from RAM to disk (sometimes called the swap).
:END:

** What's the stack?

:ANSWER:
The stack is a region of memory set aside for a thread of execution.  The stack is always LIFO order (last in first out).  So when a function is called, a block is set on top of the stack.  The block will contain memory for all the variables in the function.  When the function exits, the memory for the block on top of the stack is freed.
:END:

** What's the heap?

:ANSWER:
The heap is the region of memory set aside for a process.  A process can request more memory from the operating system, and there is no particular way the heap's memory is stored.  One can allocate an area of memory for the heap and deallocate it at any time.
:END:

** What is the [[file:~/prog/gnu/hurd/hurd-web/glibc/file_descriptor.mdwn::<braunr>%20HURD_PORT_USE%20handles%20concurrent%20access%20to%20ports][Hurd_PORT_USE]] macro for?

:answer:
It helps with accessing ports concurrently.
:END:

** What are the C symbolic constants for mach ports

:ANSWER:
MACH_PORT_RIGHT_RECEIVE
MACH_PORT_RIGHT_SEND_ONCE
:END:
** In the critique Neal mentions the trusted recover problem, what is that?
:answer:
Trust recovery, is basically turning the computer on and starting all of the passive translators.  You need to securely reset the translators to the settings that they had before the machine was shut off.  This is kind of hard to do.

Persistence is one solution, but it doesn't solve the problem completely...
[[file:~/programming/gnu/hurd/manuals/200707-walfield-critique-of-the-GNU-Hurd.pdf::%25PDF-1.4][persistance]]
:END:

** What is persistance?  ANSWER THIS ONE NEXT WEEEK
** What is the mach function to create a new port

:ANSWER:
mach_port_allocate
:END:
** If mach cannot allocate a port, how do I exit the program?

:ANSWER:
   kern_return_t err;
    mach_port_t rcv_port;

    err = mach_port_allocate( mach_task_self(), MACH_PORT_RIGHT_RECEIVE,
                              &rcv_port );

    if( err != KERN_SUCCESS ){
	    perror( "error : could not allocate any port\n" );
	    exit(err);
    }
:END:
** how do you define and declare a mach port in C?

:ANSWER:
https://www.gnu.org/software/hurd/hacking-guide/hhg.html#An-Example-using-trivfs

mach_port_t is the type.

The below is taken from a libtivfs example.
#+BEGIN_SRC C
mach_port_t bootstrap;

task_get_bootstrap_port (mach_task_self (), &bootstrap);
#+END_SRC

:END:

** What's the difference between a subhurd and a neighborhurd?

:ANSWER:
A neighbor hurd is to hurd running on the same gnumach.  Practically most computers' hardware cannot support this.

Hurd   Hurd
  \     /
   \   /
  gnumach

A subhurd is like a child hurd.  It more dependent on the parent than a neighborhurd.
:END:

** what's the differance between libnetfs and libdisfs?

:answer:
libdiskfs is great for ext2fs and related filesystems.

libnetfs is great for httpfs, ftpfs, procfs (/proc), or files in /dev
:END:

** why is it so easy for legacy applications to be fooled by a malicious file system on the Hurd?

:answer:
The Hurd allows any user to mount a filesystem.  Therefore any filesystem could be malicous.  In a unix design, all filesystems are trusted and /assumed/ to have to no bugs or viruses.  On the Hurd, this is not a safe assumption.  Somehow, the Hurd's compatibility layer must protect legacy applications.
:END:

** What is one advantage a monolithic system has over scheduling policies?

:answer:
A monolithic system can have a Big Brother watching all of the users and services.  This Big Brother can then better schedule things to be done efficiently.  On the Hurd, Mach runs various servers as userspace processes, and it does not keep much information about them.
:END:

** What is another advantage linux has over the Hurd in scheduling policies?

:answer:
Big Brother can request various subsystems of the kernel to do things that violate the formal component boundaries, but nevertheless, speed up most applications.
:END:

** What is an example of a realtime application?

:answer:
A video player.  No one wants to watch a video and have it stutter.  It needs to play smoothly!

Currently the Hurd does not have this support.  :(
:END:

** What is the nsmux translator?

:answer:
This is the namespace translator!  Super cool!
:END:

* the hurd provides mpd music player daemon!
* I should learn how to use auto tools to use gnu check
https://www.lrde.epita.fr/~adl/dl/autotools.pdf

https://libcheck.github.io/check/doc/check_html/check_3.html

https://www.sourceware.org/autobook/autobook/autobook_toc.html#SEC_Contents

https://www.gnu.org/software/autoconf/manual/

Here's how autotools basically work:

A developer writes some software and creates files named Makefile.am.  These files only describe what is necessary to build the program.  Automake takes those Makefile.am-s and generates Makefile.in files.  Then autoconf generates a configure, which turns the Makefile.in into Makefile-s.  Then it's just make && make install.

https://www.sourceware.org/autobook/autobook/autobook_11.html#Libtool-Development
In the dark the dark ages, many programs were statically linked to a library.  That means every program that wanted to use libc, had to have a copy of it!  That's a lot of many different libc-s!  Then someone figured out that it is actually possible to create a program that uses dynamic linking.  It only accesses the shared library at run time.  Now only one copy of glibc needed to be on a system!  And if an error was found in glibc, then you only need to recompile glibc, and NOT all the programs that depend on it.  But each OS has a different want to provide a linked file.  Therefore, libtool was created to handle all the various ways to handle dynamic linking on all systems.  It is related to autoconf.

I should probably have a configure.h headerfile.  These lets me programatically deal with the various differences between OSs.

In a simple autotools project, I only need to write two files

- Makefile.am   is an input to automake
  This is a highlevel, bare bones description of what is needed to build this project, and where it needs to go when it is installed.
- configure.in  is an input to autoconf
  I can use autoscan to generate a 'configure.scan'.  I'll probably have to modify configure.scan, but then I can rename it to configure.in.

  Then I run

#+BEGIN_SRC sh :results output :exports both :dir ~/prog/c/caesar
aclocal
autoreconf --install
#+END_SRC

#+RESULTS:

Then it is the usual.  There is also a needed ./compile program? automake --add-missing can help with this?

#+BEGIN_SRC sh :results output :exports both
configure
make
make install
make clean  # optional
#+END_SRC
* Hurd quiz
** True or false is Mach's IPC designed to send very large amounts of data and or streaming of data?

:answer:
True
:END:
* Downloading the irc logs

=wget --mirror --no-parent -A "hurd-*" -e robots=off http://richtlijn.be/~larstiq/hurd/=