NEW KEYBOARD CODE
-----------------

This file describes the new keyboard code which was written in late '95 for
scottb's dosemu-0.61, and adapted to the mainstream 0.63 in mid-'96.
( correction notes marked *HH  -- Hans )

What's new in the new keyboard code? A lot.

To the user:
 * Most of the keyboard-related bugs should have gone away. Hope I didn't
   introduce too many new ones (-:
   Keyboard emulation should be more accurate now; some keys are supported
   that weren't before, e.g. Pause.
 * The X { keycode } option is now obsolete. This was basically a bad hack
   to make things work, and was incompatible to X servers other than XFree86.

To the dosemu hacker:

 * While the old code already claimed to be "client-server" (and was, to
   some extent), the new code introduces a clean, well-defined interface
   between the `server', which is the interface to DOS (int9, bios etc.),
   and the `clients', which are the interfaces to the user frontends supported
   by dosemu. Currently, clients are `raw', `slang' (i.e. terminal), and `X'.
   
   Clients send keystrokes to the server through the interface mentioned
   above (which is defined in "keyboard.h"), the most important functions being
   `putkey()' and `putrawkey()'.

 * The keyboard server was rewritten from scratch, the clients were heavily
   modified.

 * There is now general and efficient support for pasting large text objects.
   Simply call paste_text().

 * The keyboard-related code is now largely confined to base/keyboard,
   rather than scattered around in various files.
   
There is a compile-time option NEW_KBD_CODE (on by default) to activate the
new keyboard code. The old stuff is still in there, but I haven't recently checked
whether it still works, or even compiles. Once the new code is reasonably well tested
I'll remove it.
( *HH: the old code _is_ made workeable and remains ON per default, it will
  stay maintained for a while, so we can easyly check where the bugs come
from )

Just like the old keyboard code, we still have the rawkeyboard=on/off and
keybint=on/off modes.


Status
------

Almost everything seems to work well now.

The keyboard server should now quite accurately emulate all key combinations
described the 'MAKE CODES' & 'SCAN CODES' tables of HelpPC 2.1, which I
used as a reference.

See below for a list of known bugs.

What I need now is YOUR beta-testing... please go ahead and try if all your
application's wierd key combinations work, and let me know if they don't.


Keyboard server interface
-------------------------

This is all you should need to know if you just want to send keystrokes
to DOS.

Use the functions

 *     putrawkey(t_rawkeycode code);
 *     putkey(Boolean make, t_keysym key)
 *     putkey_shift(Boolean make, t_keysym key, t_shiftstate shiftstate)

You may also read (but not write!) the variable 'shiftstate' if necessary.

ehm... see the DANG comments in base/newkbd-server.c for more information...

NOTE: the server's queue is limited to a relatively small number of keyboard
events (currently 15). IMO, it is not a good idea to let the queue be
arbitrarily long, as this would render the behaviour more incontrollable
if the user typed a lot of mad things while a dos program wasn't polling the
keyboard.
For pasting, there is special support in base/keyboard/keyb_clients.c which
runs on top of the server.

Keyboard server structure     
-------------------------

[NOTE: you won't need to read this unless you actually want to modify
       the keyboard server code. In that case, however, you MUST read it!]

[note: I'll have to update this. The queue backend works somewhat different
now.]

The central data structure of the keyboard server is the dosemu keyboard
queue (to be distinguished from the bios keyboard buffer, which is run
by int09 and int16).

The keyboard server code can be largely divided into the `queue frontend'
(serv_xlat.c, serv_maps.c), which does keycode translation, and the
`queue backend' (serv_backend.c, serv_8042.c), which does the interfacing
to DOS. The two sides communicate only through the queue.

Each queue entry holds a data structure corresponding to (mostly)
one keypress or release event. [The exception are the braindead
0xe02a / 0xe0aa shift key emulation codes the keyboard processor
'decorates' some kinds of keyboard events with, which for convenience
are treated as seperate events.]
Each queue entry holds a up to 4 bytes of raw keycodes for the
port 60h emulation, along with a 2-byte translated int16h keycode
and the shift state after this event was processed.
Note that the bios_key field can be empty (=0), e.g. for shift keys,
while the raw field should always contain something.

queue handling functions:

static inline Boolean queue_empty(void);
static inline void clear_queue(void);
static inline void write_queue(Bit16u bios_key,t_shiftstate shift,Bit32u raw);
static void read_queue(Bit16u *bios_key, t_shiftstate *shift, t_rawkeycode *raw);


Accordingly, the keyboard code is largely divided into two parts,
 - the 'front end' of the queue, responsible for translating keyboard
   events into the 'queue entry' format.
 - the 'back end' of the queue, which reads the queue and sends keycodes
   to DOS




THE FRONT END
-------------


 putrawkey() -------->----+
      \   \               |
       \   v              |
        \  translate()    |
         \     |          |
          \    v          \    (t_rawkeycode[4])      /---QUEUE----\
     /->---\---|-----------*------------------------> [ raw        ]
    /       \  \  (t_keysym+char)                     [            ]
 putkey() ->-\--*--------------> make_bios_code() --> [ bios_key   ]
    \         \                                       [            ]
     \         v                           /--------> [ shiftstate ]
      \---> do_shift_keys()               /           \------------/
                |                        /
                v        (t_shiftstate) /
            [shiftstate]---------------/


--------->  data flow (&calls, sometimes)
.........>  calls


Functions in serv_xlat.c:
-----------------------------

static Boolean do_shift_keys(Boolean make, t_keysym key);
static Bit16u make_bios_code(Boolean make, t_keysym key, uchar ascii);
static uchar translate(t_keysym key);
static Boolean handle_dosemu_keys(t_keysym key);
void putrawkey(t_rawkeycode code);
void putkey(Boolean make, t_keysym key, uchar ascii);
void putkey_shift(Boolean make, t_keysym key, uchar ascii, t_shiftstate s);


Any keyboard client or other part of dosemu wishing to send keyboard
events to DOS will do so by calling one of the functions putrawkey,
putkey, and putkey_shift.


putrawkey
---------

   is called with a single raw scancode byte. Scancodes from subsequent
   calls are assembled into complete keyboard events, translated and
   placed into the queue.

putkey & others
------
   ...to be documented.


THE BACK END
------------


QUEUE BACK END in keybint=on mode
---------------------------------

                   EMULATOR SIDE        |    x86 SIDE
		                        |
                      ....[through PIC].|....................
                      :                 |           :        v
QUEUE      .....> out_b_8042() --> [ port 60h ] ----:---> other_int9_handler
|         :                             |        \  `.......    (:) (|)
|         :                             |         \         v   (v) (|)
+->int_chk_q()-> bios_buffer----> [ get_bios_key ]-----> default_int9_handler
      ^  \                           :  |                   |       (|)
      :   \----> shiftstate_buffer   :  |                   v       (v)
      :               |         .....:  |               bios keyb buffer
      :               v        v        |
      :          copy_shift_state() ----+-------------> bios shiftstate
      :                                 |
      :                                 |
      :                                 |
    backend_run()                       |



Abbreviations:
int_chk_q() = int_check_queue()
out_b_8042() = output_byte_8042()


QUEUE BACK END in keybint=off mode
----------------------------------

                  EMULATOR SIDE         |    x86 SIDE
                                        |
             kbd_process()              |
                  :     :               |
                  :     v               |
 QUEUE -----------:--> put_keybuf() ----+-------------> bios keyb buffer
     \            v                     |
      \--------> copy_shift_state() ----+-------------> bios shiftstate
                                        |
                                        |   
                                        |



Functions in newkbd-server.c
----------------------------

void do_irq1();
void clear_keybuf();
static inline Boolean keybuf_full(void);
static inline void put_keybuf(Bit16u scancode);
void copy_shift_state(t_shiftstate shift);
static void kbd_process(void);


Transfer of the keyboard events from the dosemu queue to DOS is done in
two completely different ways, depending on the keybint setting in
dosemu.conf:

   (1) keybint=off
       kbd_process() simply reads the queue until it finds a bios keycode
       (as we're not interested in raw scancodes without int9 emulation),
       which it stores in the bios keyboard buffer, while also copying
       the shift state to the appropriate bios variables.

   (2) keybint=on
       As soon as a key is stored into the empty queue, kbd_process() triggers
       IRQ1 through the PIC emulation, which some time later will call do_irq1().
       
       do_irq1() will prepare for the interrupt execution by reading from
       the queue and storing the values in the variables raw_buffer,
       shiftstate_buffer, and bios_buffer, and then call run_irq() to
       run the actual DOS interrupt handler.

       again, there are two cases:
       - the default int09 handler in the dosemu bios (base/bios_emu.S)
         will call the helper function get_bios_key(), which returns
         the translated bios keycode from bios_buffer and copies the
         shiftstate from shiftstate_buffer. The raw keycodes are not used.
         get_bios_key() may also return 0 if no translated keycode is
         ready.
	 The int9 handler will also call the `keyboard hook' int15h, ax=????.

       - if a dos application or TSR has redirected the keyboard interrupt,
         its handler might read from port 60h to get raw scancodes.
         Port 60h is of course virtualized, and the read returns the value
         from raw_buffer.

       Note that a mix between the two cases is also possible, e.g. a
       TSR's int9 handler first reads port 60h to check if a particular
       key was pressed, then gives over to the default int9 handler.
       Even these cases should be (and are, I think) handled properly.

       Note also that in any case, int9 is called once for each raw scancode
       byte. Eg.,suppose the user pressed the PgDn key, whose raw
       scancode is E0 51:

       - first call to int9:
             read port 60h        = 0xe0
             read port 60h        = 0xe0   (**)
             call get_bios_key()  = 0
             iret
         do_irq1() reschedules IRQ1 because further scancodes are in the queue
       - second call to int9
             read port 60h        = 0x51
             call get_bios_key()  = 0x5100    (bios scancode of PgDn)
             iret
     
       (** multiple port 60h reads during the same interrupt yield the
           same result.)


This is not a complete documentation. If you actually want to hack the 
keyboard server, you can't avoid reading the code, I'm afraid ;-)


Known bugs & incompatibilites
-----------------------------

* behaviour wrt. cli/sti is inaccurate, because the PIC code currently
  doesn't allow un-requesting if IRQ's.

* emulation of special 8042 and keyboard commands is incomplete and
  probably still somewhat faulty.
  
* the 'internal' keyboard flags in seg 0x40, like E0 prefix received etc.
  are never set. This shouldn't hurt, for all but the most braindead
  TSRs.

* CAPS LOCK uppercase translation may be incorrect for some (non-german)
  national characters.

* typematic codes in X and non-raw modes are Make+Break, not just Make.
  This shouldn't hurt, though.

* in X mode, shift+Gray cursor keys deliver numbers if NumLock is off.
  This is an X problem, and AFIK nothing can be done about it.

* in X, something may be wrong with F11+F12 handling (and again, possibly
  necessarily wrong).

* the Pause key works in terms of raw scancodes, however it's function
  is not implemented (i.e. it doesn't actually halt DOS execution.)

* in terminal (i.e. slang) mode, several things might be wrong or at least
  improveable.

* there is no difference between the int16h functions 0,1 and the extended
  functions 0x10,0x11 - i.e. 0,1 don't filter out extended keycodes.

* keyb.exe still doesn't work (hangs) - most probably due to the above.


Changes from 0.61.10
--------------------

* adapted to 0.63.55

* adapted to 0.63.33

* renamed various files

* various minor cleanups

* removed putkey_shift, added set_shiftstate

* in RAW mode, read current shiftstate at startup

* created base/keyboard/keyb_client.c for general client initialisation and
  paste support.


TODO
----

find what's wrong with TC++ 1.0
implement pause key
adapt x2dos (implement interface to send keystrokes from outside dosemu)
once everything is proved to work, remove the old keyboard code


---------------------------------------------------------------------------
R.Zimmermann <zimmerm@mathematik.uni-marburg.de>      Last updated 18 Sep 96
                                    update from Hans:              17 Jan 97
