TITLE

Parrot JIT Subsystem

VERSION

CURRENT

Maintainer: Daniel Grunblatt 
Class: Internals
PDD Number: 8
Version: 1.2
Status: Developing
Last Modified: 31 January 2002
PDD Format: 1
Language:English

ABSTRACT

This PDD describes the Parrot Just In Time compilation subsystem.

DESCRIPTION

The Just In Time, or JIT, subsystem converts a bytecode file to native machine code instructions and executes the generated instruction sequence directly.

IMPLEMENTATION

Currently works on Intel x86, ALPHA, and SPARC version 8 processor systems, on most operating systems. Currently only 32-bit INTVALs are supported.

The initial step in generating native code is to invoke Parrot_jit_begin, which generally provides architecture specific preamble code. For each parrot opcode in the bytecode, either a generic or opcode specific sequence of native code is generated. The .jit files provide functions that generate native code for specific opcode functions, for a given instruction set architecture. If a function is not provided for a specific opcode, a generic sequence of native code is output which calls the interpreter C function that implements the opcode. Such opcode are handled by Parrot_jit_normal_op.

If the opcode can cause a control flow change, as in the case of a branch or call opcode, an extended or modified version of this generic code is used that tracks changes in the bytecode program counter with changes in the hardware program counter. This type of opcode is handled by Parrot_jit_cpcf_op.

While generating native code, certain offsets and absolute addresses may not be available. This occurs with forward opcode branches, as the native code corresponding to the branch target has not yet been generated. On some platforms, function calls are performed using program-counter relative addresses. Since the location of the buffer holding the native code may move as code is generated (due to growing of the buffer), these relative addresses may only be calculated once the buffer is guaranteed to no longer move. To handle these instances, the JIT subsystem uses "fixups", which record locations in native code where adjustments to the native code are required.

FILES

jit/${jitcpuarch}/jit_emit.h

This file defines Parrot_jit_begin, Parrot_jit_dofixup, Parrot_jit_normal_op and Parrot_jit_cpcf_op. In addition, this file defines the macros and static functions used in .jit files to produce binary representations of native instructions.

jit/${jitcpuarch}/core.jit

The functions to generate native code for core parrot opcodes are specified here. To simplify the maintenance of these functions, they are specified in a format that is pre-processed by jit2h.pl to produce a valid C source file, jit_cpu.c. See "Format of .jit Files" below.

jit/${jitcpuarch}/string.jit

The string subsystem.

include/parrot/jit.h

This file contains definitions of generic structures used by the JIT subsystem.

The op_jit array of jit_fn_info_t structures, provides for each opcode, a pointer to the function that generates native code for the opcode, whether the generic Parrot_jit_normal_op or Parrot_jit_cpcf_op functions or an opcode specific function.

The Parrot_jit_fixup structure records the offset in native code where a fixup must be applied, the type of fixup required and the specific information needed to perform the parameters of the fixup. Currently, a fixup parameter is either an opcode_t value or a function pointer.

The Parrot_jit_info structure holds data used while producing and executing native code. An important piece of data in this structure is the op_map array, which maps from opcode addresses to native code addresses.

jit.c

build_asm() is the main routine of the code generator, which loops over the parrot bytecode, calling the code generating routines for each opcode while filling in the op_map array. This array is used by the JIT subsystem to perform certain types of fixups on native code, as well as by the native code itself to convert bytecode program counters values (opcode_t *'s) to hardware program counter values.

The bytecode is considered an array of opcode_t sized elements, with parallel entries in op_map. op_map is initially populated with the offsets into the native code corresponding to the opcodes in the bytecode. Once code generation is complete and fixups have been applied, the native code offsets are converted to absolute addresses. This trades the low up-front cost of converting all offsets once, for the unknown cost of repeatedly converting these offsets while executing native code.

jit2h.pl

Preprocesses the .jit files to produce and prints the struct opcode_assembly_t.

Format of .jit Files

Jit files are interpreted as follows:

op-name { body }

Where op-name is the name of the Parrot opcode, and body consists of C syntax code which may contain any of the identifiers listed in the following section.

Identifiers

In general, prefixing an identifier with & yields the address of the referenced Parrot register or constant. If an identifier is given without a prefix, the value is returned by default. To emphasis the use of the value, the * prefix may be used. Since Parrot register values vary during code execution, their values can not be obtained through identifier substitution alone.

INT_REG[n]

Gets replaced by the INTVAL register specified in the nth argument.

NUM_REG[n]

Gets replaced by the FLOATVAL register specified in the nth argument.

STRING_REG[n]

Gets replaced by the STRING register specified in the nth argument.

INT_CONST[n]

Gets replaced by the INTVAL constant specified in the nth argument.

NUM_CONST[n]

Gets replaced by the FLOATVAL constant specified in the nth argument.

STRING_CONST_strstart[n]

Gets replaced by strstart of the STRING constant specified in the nth argument.

STRING_CONST_buflen[n]

Gets replaced by buflen of the STRING constant specified in the nth argument.

STRING_CONST_flags[n]

Gets replaced by flags of the STRING constant specified in the nth argument.

STRING_CONST_strlen[n]

Gets replaced by strlen of the STRING constant specified in the nth argument.

STRING_CONST_encoding[n]

Gets replaced by encoding of the STRING constant specified in the nth argument.

STRING_CONST_type[n]

Gets replaced by type of the STRING constant specified in the nth argument.

STRING_CONST_language[n]

Gets replaced by language of the STRING constant specified in the nth argument.

CONST_INT[n]

Gets replaced by the nth integer constant defined in jit.c

CONST_FLOAT[n]

Gets replaced by the nth floatval constant defined in jit.c

CONST_CHAR[n]

Gets replaced by the nth char constant defined in jit.c

TEMP_INT[n]

Gets replaced by the nth temporary integer array.

TEMP_FLOAT[n]

Gets replaced by the nth temporary float array.

TEMP_CHAR[n]

Gets replaced by the nth temporary char array.

&INTERPRETER[n]

Gets replaced by the address of the interpreter.

*CUR_OPCODE[n]

Gets replaced by the address of the current opcode in the Parrot bytecode.

FUNC(func-name, arg1, ..., argN)

Call a function defined in another .jit file (except from the core).

SYSTEM_CALL(syscall-name, arg1, ..., argN)

Call a system call.

CALL(func-name, arg1, ..., argN)

Call a C function. The idea is to replace all the CALL() with FUNC().

Arguments to CALL and SYSTEMCALL

The arguments to CALL and SYSTEMCALL must be preceeded by V to indicate that the value should be taken as an immediate or A to indicate that the value should be dereferenced.

ALPHA Notes

The access to Parrot registers is done relative to $6, all other memory access is done relative to $27, to access float constants relative to $7 so you must preside the instruction with ldah $7,0($27).

EXAMPLE

Let's see how this work:

Parrot Assembly:

set I0,8
set I2,I0
print "Big piece of JIT\n"
time I0
end

Parrot Bytecode: (only the bytecode segment is showed)

+-----------------------------------------------+
| 63 | 0 | 8 | 62 | 2 | 0 | 24 | 0 | 48 | 0 | 0 |  
+-|------------|------------|--------|--------|-+
  |            |            |        |        |
  |            |            |        |        +-- end (no arguments)
  |            |            |        +----------- time_i (1 argument)
  |            |            +-------------------- print_sc (1 argument)
  |            +--------------------------------- set_i_i (2 arguments)
  +---------------------------------------------- set_i_ic (2 arguments)

Please note that the opcode numbers used might have already changed.

Intel x86 assembly version of the Parrot ops:

Parrot_set_i_ic {
   movl *INT_CONST[2],&INT_REG[1]
}

Parrot_set_i_i {
   movl &INT_REG[2],%eax
   movl %eax,&INT_REG[1]
}

Parrot_print_sc {
   movl $1,&TEMP_INT[1]
   SYSTEMCALL(WRITE,3, A&TEMP_INT[1] V&STRING_CONST_strstart[1] V*STRING_CONST_strlen[1])
}

Parrot_end {
   leave
   ret
}

Note that there is no Parrot_time_i so, the code generated by the C compiler for Parrot_time_i will be called.

Intel x86 object code of the Parrot ops:

Parrot_set_i_ic {
   \xc7\x05\x00\x00\x00\x00\x00\x00\x00\x00 # mov $0,0x0
}

Parrot_set_i_i {
   \xa1\x00\x00\x00\x00                     # mov 0x0,%eax
   \xa3\x00\x00\x00\x00                     # mov %eax,0x0
}

Parrot_print_sc {
   \xc7\x05\x00\x00\x00\x00\x01\x00\x00\x00 # mov $1,0x0
   \x68\x00\x00\x00\x00                     # push 0x0
   \x68\x00\x00\x00\x00                     # push 0x0
   \xff\x35\x00\x00\x00\x00                 # push $0
   \x50                                     # push 
   \xb8\x04\x00\x00\x00                     # mov $4,%eax
   \xcd\x80                                 # int 80h
   \x72\x00                                 # jb 0
}

Parrot_end {
   \xc9                                     # leave
   \xc3                                     # ret
} 

Parrot_time_i {
   \x68\x00\x00\x00\x00                     # pushl 0x0
   \x68\x00\x00\x00\x00                     # pushl 0x0
   \xe8\x00\x00\x00\x00                     # call 0x0
   \x83\xc4\x08                             # add $0x8,%esp
}

The object code for time_i is the same that for any opcode that isn't implemented in core.jit

Build process:

Memory dump of the JIT code being generated:

 +-----------------------------------------+
 | 0x55 0x89 0xe5 0xc7 0x05 0x00 0x00 0x00 |
 | 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 |
 +-----------------------------------------+

That is the state after the code for the first op has been copied. The 0x55 0x89 0xe5 you see before the object code for Parrot_set_i_ic is the output of Parrot::Jit->init()

Fill it with addresses and/or values:

 +-----------------------------------------+
 | 0x55 0x89 0xe5 0xc7 0x05 0x00 0xa0 0x10 |
 | 0x00 0x08 0x00 0x00 0x00 0x00 0x00 0x00 |
 +-----------------------------------------+

The address of I0 (&intepreter->int_reg.registers[0]) is 0x10a000 (or whatever), so the first 4 bytes after the opcode number are filled with it, and the other contiguous 4 with the constant it self.

The same process is done one time per opcode.

The final result:

 +-----------------------------------------+
 | 0x55 0x89 0xe5 0xc7 0x05 0x00 0xa0 0x10 |
 | 0x00 0x08 0x00 0x00 0x00 0xa1 0x00 0xa0 |
 | 0x10 0x00 0xa3 0x08 0xa0 0x10 0x00 0xc7 |
 | 0x05 0x54 0x7a 0x10 0x00 0x01 0x00 0x00 |
 | 0x00 0x68 0x11 0x00 0x00 0x00 0x68 0x18 | 
 | 0xb0 0x10 0x00 0xff 0x35 0x54 0x7a 0x10 |
 | 0x00 0x50 0xb8 0x04 0x00 0x00 0x00 0xcd |
 | 0x80 0x72 0x00 0x68 0x00 0xa0 0x10 0x00 |
 | 0x68 0xe0 0x60 0x12 0x00 0xe8 0xae 0xdb |
 | 0xed 0xff 0x83 0xc4 0x08 0xc9 0xc3 0x00 |
 +-----------------------------------------+

This code is ready to be called.

To install Make, copy and paste the appropriate command in to your terminal.

cpanm

cpanm Make

CPAN shell

perl -MCPAN -e shell
install Make

For more information on module installation, please visit the detailed CPAN module installation guide.

	Global
`s`	Focus search bar
`?`	Bring up this help dialog

	GitHub
`g` `p`	Go to pull requests
`g` `i`	go to github issues (only if github is preferred repository)

	POD
`g` `a`	Go to author
`g` `c`	Go to changes
`g` `i`	Go to issues
`g` `d`	Go to dist
`g` `r`	Go to repository/SCM
`g` `s`	Go to source
`g` `b`	Go to file browse

	Search terms
module: (e.g. module:Plugin)
distribution: (e.g. distribution:Dancer auth)
author: (e.g. author:SONGMU Redis)
version: (e.g. version:1.00)