README 36.1 KB
Newer Older
1
2
3
4
What is Cexp?
-------------

 -- Brought to you by Till Straumann <strauman@slac.stanford.edu> --
5
 -- $Id$ --
6

7
The Cexp utility is a C-expression interpreter which
strauman's avatar
strauman committed
8
9
10
11
gives its user access to all symbols present in the currently
executing program. Its primary target is RTEMS, an open source
RTOS, but Cexp should compile and run on virtually any platform
supported by the BFD library.
12
13
14
15
16
17

If Cexp is linked to an executable, it is
possible to invoke arbitrary functions and to
read/write variables (and hence virtually any
memory location) by interpreting C-style expressions.

18
19
20
Cexp can access object files using either the pmbfd/pmelf
libraries ('poor-man's BFD) that comes with Cexp or libbfd.
The latter is more powerful:
till's avatar
till committed
21
 - not limited to the ELF object format.
22
 - supports many more machine types (pmbfd currently 
strauman's avatar
strauman committed
23
24
25
   only supports 32-bit powerpc, m68k, i386, x86_64
   and sparc [32-bit] -- in decreasing order of testing
   that has been done).
till's avatar
till committed
26
 - using BFD / libopcodes, a disassembler comes for free.
27
28
29
30
31
32

Otoh, using pmbfd/pmelf has the following advantages:
 - much smaller memory footprint.
 - less restrictive license. Linking against the BFD
   library is subject to the GPL. See LICENSING
   below for more information.
till's avatar
till committed
33

till's avatar
till committed
34
35
36
What is New?
------------

37
38
 - for later versions, consult the ChangeLog file.

39
40
41
42
 - 2003/02/14: CEXP_1.2.beta ("Valentine") released.
   It supports G++ prioritized execution of static
   constructors/destructors.

till's avatar
till committed
43
44
45
46
47
48
 - 2003/02/13: CEXP_1.1.beta is released. It fixes a bug
   of the exception handler registration.
   
   Makefile.syms has been added and modularized copies
   of the RTEMS 'cdtest' sample program.

49
50
51
52
53
54
55
56
Description
-----------

CEXP is a simple utility featuring
 - symbol table
 - C expression parser/interpreter
 - type "engine"
 - user definable variables
57
58
59
 - recursive invocation/simple scripts
 - runtime 'module' loader/linker
 - disassembler (only if linked with BFD)
strauman's avatar
strauman committed
60
 - CEXP is reentrant
61
62
63
64
65
66

Cexp knows about the basic C types (no aggregates), i.e.

char,   char*
short,  short*
long,   long*
67
float,  float*
68
69
70
71
72
73
double, double*
long    (*)()
double  (*)()

and interprets C-style expressions, e.g.

till's avatar
till committed
74
  Cexp> printf("Hello world")
75

till's avatar
till committed
76
  Cexp> some_variable = 0xdeadbeef
77

till's avatar
till committed
78
79
80
  Cexp> some_double_variable = *(double*)&some_variable

  Cexp> a=printf, a && a("The square root of 2 is %g\n",sqrt(2.0))
81
82
83

Symbol Table
------------
84
85
86
A) Using a Symbol Table File
- - - - - - - - - - - - - - -
On startup, Cexp reads the 'system' symbol table either from 
strauman's avatar
strauman committed
87
88
the executable itself, or from a stripped-down '.sym'
file - such a stripped-down symbol file can be created using
89
90
the 'objcopy' (preferred, but needs binutils 2.18 or later)
or 'xsyms' (deprecated; comes with Cexp) utilities.
91
92
93

B) Builtin Symbol Table File
- - - - - - - - - - - - - - -
94
The 'xsyms' tool can be used to generate
95
96
97
98
a symbol table in C-source form for subsequent compilation and
linkage into the final executable. This usually involves linking
the executable twice:
  1) compile and link all sources, libraries etc.; build an executable
99
  2) xsyms -C executable  mysymtab.c
100
101
  3) compile mysymtab.c (set -I to Cexp installation 'include' directory
     or Cexp source directory, since <cexpsyms.h> is included)
102
  4) link application again, but this time add 'mysymtab.c' to the
103
     list of sources.
104
105
106
107
108
109
110
111
112
113
114
115
  NOTES: Step 1) linking succeeds without a symtab.o because there's
         a weak NULL-ptr alias for the builtin symtab.
		 When building the demo program you can use a builtin symbol
		 table:
		   a) build demo the normal way (corresponds to step 1 above)
		   b) type 'make builtin-symtab'
		      this extracts symbol table info from the executable
			  created in step a) and creates an object file
			  'cexp-builtin-symtab.o' (steps 2+3 above).
		   c) type 'make' again. This re-links the demo program
		      now using the 'cexp-builtin-symtab.o' object
			  (corresponds to step 4 above).
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
A similar source file can also be generated using the 'ldep' utility
(discussed elsewhere, it is part of RTEMS-GeSys).

Without access to a symbol table file which also provides Cexp
with information about the architecture it is executing on, Cexp
might fail to detect the correct BFD CPU architecture which is
needed for the disassembler. In such cases, the '-a' command
line option can be used to specify the architecture (arch string
can be obtained by running 'objdump -f' on the ELF executable).

Symbol Type
- - - - - -

Once the symbol table is read, Cexp guesses the symbol/variable
types from the symbol sizes. (Guess is only supported on ELF targets.)
till's avatar
till committed
131
132

In some cases, this guess is obviously wrong, but you can 
till's avatar
till committed
133
always cast the symbol to the correct type.
strauman's avatar
strauman committed
134

135
Arrays are normally 'void' like every symbol for which's
strauman's avatar
strauman committed
136
type Cexp cannot make a guess. It is still possible, however, to
137
138
139
use such a symbol, simply by taking its address and
casting to a different pointer type:
    
till's avatar
till committed
140
Cexp> *(((long*)&some_array)+25)
141
142

evaluates to the unsigned long array element
till's avatar
till committed
143
some_array[25]. Alternatively, you can redeclare
till's avatar
till committed
144
its type:
till's avatar
till committed
145

146
Cexp> long *some_array !
till's avatar
till committed
147
Cexp> *(some_array + 25)
148

149
150
151
152
153
Note the exclamation mark: terminating a variable declaration with an
exclamation mark allows you to redefine its type. Only new variables
may be declared without the '!'. This protects you from erroneously
use existing symbols.

154
155
156
157
Cexp provides a series of lookup functions
for symbols. Most noteworthy:

lkup(char *regexp)
till's avatar
till committed
158
lkaddr(unsigned long addr)
159
160
161
162
163
164
165
166

These are just normal C functions which can be
invoked by Cexp. Note that lkup() takes a regular
expression argument allowing for powerful searching.

C-Expression Parser / Interpreter
---------------------------------
The C expression parser has some restrictions:
till's avatar
till committed
167
168
169
170
171
172
  - no '?' ':' expression
  - no [], '->'
  - the '.' operator has a special meaning - see below
  - no multilevel pointer dereferencing (since e.g. char**
    is not a valid type, Cexp cannot dereference 
    **addr. You must explicitly cast this:
173

till's avatar
till committed
174
      Cexp> *(char*)*(long*)some_char_pointer_pointer
175

till's avatar
till committed
176
177
which assumes that sizeof(long) == sizeof(char*)
for the particular machine.
178
179
180
181
182
183

However, most of the other operators are available, including
the 'comma' operator and the logical '&&' and '||' with the
correct semantics, i.e. conditional evaluation of expression
parts e.g.

till's avatar
till committed
184
  Cexp> (dd=deviceDetect(_some_device_address)) && driverInit(dd)
185
186
187
188

will call the driverInit() function only if deviceDetect()
returns a nonzero value.

till's avatar
till committed
189
190
191
192
193
The '.' operator (structure field access) has a special
meaning to Cexp: symbol "member" (in an OOP sense) access.
Currently, the only member function defined is 'help'. Consult
the 'Help' section about details.

194
195
Function calling:
The user is responsible for feeding properly typed arguments;
strauman's avatar
strauman committed
196
unused arguments will be filled with integral/scalar 0, which on 
197
most ABIs is safe.
till's avatar
till committed
198
Since the symbol table (on ELF, Cexp is not using .stab but .symtab)
199
200
201
202
provides no info about a function's return type, Cexp assumes
all functions to return long. Floating point functions
must be cast appropriately:

till's avatar
till committed
203
204
205
206
207
208
  Cexp> ((double(*)())sqrt)(2.345)

Or

  Cexp> double (*sqrt)()
  Cexp> sqrt(2.345)
209

strauman's avatar
strauman committed
210
211
A lot of type casting can be avoided by re-declaring or
using user defined variables, see below.
212

till's avatar
till committed
213

214
215
216
217
218
219
220
221
222
223
Type Engine
-----------
Cexp knows about (and only about) the primitive types listed
above.
NOTE: Cexp treats ALL integral types as UNSIGNED although
no 'unsigned' keyword is used or recognized. The main
reason for this is to save typing. Hence
char is in fact unsigned char etc.

An 'int' type is missing (but could easily added) which
224
means that on some machines (e.g. 64-bit cpus) no 32bit
225
226
227
228
229
230
231
232
type is currently available...

User Variables
--------------
Cexp supports (fully typed) user variables which are
visible in a global namespace (i.e. visible to other
instances of Cexp). The name of user variables must
not collide with symbols present in the symbol table.
strauman's avatar
strauman committed
233
234
(When loading object files [AKA 'modules'], name clashes
are ignored. A symbol in a newly loaded module that
235
236
conflicts with an already defined user variable silently
takes precedence [object file symbols have a higher
strauman's avatar
strauman committed
237
238
239
priority than user variables]. However, after unloading
the conflicting module, the user variable becomes again
'visible'.)
240
241
242
243
244
245
246
247
248
249
250
251
252

A user variable is created simply by assigning it
a value which also automatically defines its type.

    hallo="hallo"

creates a 'char*' and assigns it the address of a
string constant. The string is stored in 'malloc()ated'
memory and 'lives' forever. Subsequent use of the
same string (strcmp(a,b)==0) results in re-using the
already stored instance of the string.
String constants must not be written-to!

253
254
255
256
257
258
259
260
261
262
Alternatively, a user variable can be introduced
by declaring it:

   char *hallo

This method is useful if you use identifiers that
could already exist in the application's symbol table
-- if this is the case then the declaration will fail
and thus warn you.

263
The type of user variables can be modifed, simply
264
265
266
267
268
by re-declaring them, in a C-style manner (the
exclamation mark actually deviates from strict
C syntax. It is required when re-declaring to
avoid accidential re-declaring of existing
variables):
269

270
    long hallo !
271
272
273
274
275
276
277
278
279
280
281

results in 'hallo' maintaining its value (the address
of the string constant) but interpreting it as a long.
Hence

    chppt = &hallo

is an 'disguised' char** and

    *(char*)*chppt

strauman's avatar
strauman committed
282
yields 'h'
283

strauman's avatar
strauman committed
284
285
User variables can also hold function pointers and hence
can be handy abbreviations for long names and casts.
286
287
288
289
290
291
292
293
294
295
E.g. a convenient variable can be set for 'sqrt()'
(which returns a double value):

    s=((double(*)())sqrt)

It is then possible to automatically get the correct
return type:

    printf("Let's print a double: %g\n",s(44.34))

296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
The special user variable 'ans' stores the result
of the last successfully evaluated expression. E.g:

    Cexp> 3+4
	0x00000007 (7)
	Cexp> ++ans
	0x00000008 (8)
	Cexp> printf("%u\n",ans)
	8
	0x00000002 (2)
	Cexp>

The 'ans' variable is special in the sense that unlike
other user variables it is not global but local to the
executing parser, i.e., parsers running in different
task contexts each maintain their own instances of
'ans'.

till's avatar
till committed
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
Help Facility
-------------

The '.' operator (structure field access) has a special
meaning to Cexp: symbol "member" (in an OOP sense) access.
Currently, the only member function defined is 'help'. I.e.
by typing

Cexp>  some_symbol.help()

You get help information about that symbol. If no help
is defined, or if you request 'verbose' help:

Cexp> some_symbol.help(1)

till's avatar
till committed
329
the symbol address, value and type information are printed
till's avatar
till committed
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
(same format as the 'lkup' output).

You can add/modify help information to a symbol (e.g. a user
defined variable) simply by providing a string argument to
the symbol's help member:

Cexp> long myvar
Cexp> myvar.help("I just created a 'long' variable")

Note that help is stored 'per symbol', i.e. when creating
an 'alias' as in the example above:

Cexp> s=(double (*)())sqrt

the help information is not propagated to 's'. If 'sqrt'
strauman's avatar
strauman committed
345
had help information, it would have to be copied:
till's avatar
till committed
346
347
348

Cexp> s.help(sqrt.help())

till's avatar
till committed
349
350
351
This example shows that the return value of the 'help() member'
is the address of the (static) help text string.

352
353
354
355
356
357
358
359
How to Start/Invoke Cexp - Simple Scripts
-----------------------------------------
Cexp has two entry points,

    cexp_main(int argc, char **argv)

and

360
    cexpsh(char *arg1,...)
361
362
363
364

i.e. a 'main' style and a 'vararg' style. The
'vararg' version ends up building an argument list
and calling cexp_main(). It is mainly intended
365
for recursively invoking 'cexpsh()' e.g. for 
366
367
reading a series of lines from a file.
Note that the string argument list submitted to
368
369
cexpsh() must be NULL terminated. When calling
cexpsh() from cexpsh() (e.g. to interpret a script),
370
this is not necessary, however, because any unused
strauman's avatar
strauman committed
371
372
arguments (up to the internal maximum of 10 args)
are filled with zeroes...
373

374
The calling syntax of cexp_main/cexpsh is as follows:
375

376
cexpsh [-s <symbolfile>] [-c <expression>] [-h] [-d] [-q] [<scriptfile>]
377
378
379

-h and -d have the obvious effect of printing usage
info and enabling debugging information (only available
strauman's avatar
strauman committed
380
if configured with --enable-YYDEBUG).
381

382
The -q ('quiet') flag instructs Cexp not to print
till's avatar
till committed
383
384
any normal output (errors are still reported to stderr)
on stdout. This behavior can be desirable when evaluating
385
386
387
388
389
scripts. Note, however, that this affects only output
generated by Cexp itself; output produced by functions
which are called during expression evaluation is 
unaffected by the '-q' flag. Consult the section about
'Redirection' for further information.
till's avatar
till committed
390

391
392
Note that (unless you are using a built-in symbol
table) the -s option MUST be used the first time
393
394
395
396
397
398
399
400
401
402
403
'Cexp' is started and it must be provided with an
appropriate symbol file. This can be the executable
itself of a stripped version (use the 'xsyms' utility)
to reduce memory usage and loading time on RTEMS systems.
A basic check is made to protect against version mismatch
between the symbol file and the executable.

Once Cexp has loaded the system's symbol table, further
instances will simply use the global system table if
the -s argument is missing.

404
Cexp then reads commands from stdin (using the TECLA
405
406
library) or alternatively from a script file or
from a string argument (-c option).
407
408

Cexp ignores any characters present on a line
strauman's avatar
strauman committed
409
after it scans the comment tokens
410
411
412
413
414
'#' or '//'.

Example for invoking Cexp from Cexp to evaluate
a script:

415
  Cexp> cexpsh("script_file")
till's avatar
till committed
416

417
418
419
While the former syntax involves recursive execution of
Cexp, it is now also possible to let the current instance
divert its input to read from a script using the
420
'.' operator:
421

422
  Cexp> . script_file
423
424
425
426
427
428
429
430
431
432
433
434

There are slight syntactic differences when using this syntax,
though. Evaluation of the file name is not performed by
the parser but by a simple preprocessor. The filename doesn't have
to be embedded in double quotes (but it can be). Note that
no escape sequences are recognized. It is possible, however,
to use filenames with embedded whitespace by enclosing the
script name in single or double quotes. In this case, the
entire string between matching quotes is used. E.g., (RHS
string delimiter is '"' -- remember that there are no escape
sequences...):

435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
  . a b c     --> read from file "a"
  . 'a b' "c" --> read from file "a b"
  . "a'b"     --> read from file "a'b"
  . "a\"b"    --> read from file "a\\" (a followed by a single backslash)

Note that the '. infile' notation differs from 'true'
stdin redirection (as described under 'Redirection'). The
functionality described here instructs the interpreter to
read expressions from a file. However, the 'stdin' stream
used by any functions that are called during expression
evaluation still is unchanged (i.e., the console).

Redirection
-----------
The stdio streams (and only these -- the underlying file
descriptors are unchanged; in a multithreaded system like
RTEMS file descriptors are global entities anyways whereas the
stdio streams are 'per-thread' objects so that it makes
more sense to redirect these rather than descriptors)
used during expression evaluation may be redirected to
arbitrary files.
The syntax is similar to redirection under 'csh' but the
redirection operators must *precede* the expression that
is to be evaluated (this is necessary because e.g.,

       printf("Hello\n") > "some_file"

would be a valid C-expression (comparison of return
value of 'printf()' with character pointer).

Hence, the following syntax was chosen:

cexp_input_line:  [ redirectors ]   expression

redirectors: inp_redir | out_redir | inp_redir out_redir | out_redir inp_redir

out_redir:  ( '>' | '>>' | '>&' | '>>&' ) redirarg ':'
inp_redir:    '<'                         redirarg ':'

redirarg:  <string_constant> | <string_variable>

Examples:

a) Redirect output of expression 'printf("Hello\n")' to file "hellof"

   Cexp> > "hellof" : printf("Hello\n")

b) Use a string variable to store path to a file, redirect
   stdin so that 'scanf' reads a string from the file.

   #allocate a buffer
   Cexp>  buf    = malloc(1000)
   0x00031423 (201763)
   Cexp>  mypath = "/a/b/c/d/infile"
   0x00040567 (263527)
   Cexp> < mypath : scanf("%s",buf)
   0x00000001 (1)

c) Append first string from 'infile' to 'outfile'

   Cexp> < mypath : >> "outfile : scanf("%s",buf), printf("%s",buf)

Output redirection operators
- - - - - - - - - - - - - - -
As in 'csh' the operators have the following semantics:
  '>'   truncate file and redirect stdout to it
  '>&'  truncate file and redirect both, stdout and stderr to it
  '>>'  open or create file and redirect stdout to append to the file
  '>>&' open or create file and redirect both, stdout and stderr
        to append to the file

Note that it is not directly possible to redirect stderr independently
from stdout but the standard csh workaround can be used: Redirect
stdout of a subshell to file 'a' and redirect stdout+stderr of
the parent to file 'b':

   Cexp> >& "errfile" : cexpsh("-c","> \"outfile\" : <quoted_expression>")

Note regarding input redirection
- - - - - - - - - - - - - - - - -
Note that the '<' operator can have two different meanings:

 < "filename"

instructs the interpreter to read expressions from "filename"
(this syntax is equivalent to '. "filename"' but DEPRECATED
-- use '.' to 'source' scripts) whereas

 < "filename" : <expression>
524

525
redirects 'stdin' during the evaluation of <expression>
till's avatar
till committed
526

till's avatar
till committed
527
528
529
Loadable Modules / Runtime Loader
---------------------------------

till's avatar
till committed
530
When built aginst the pmbfd or BFD library, Cexp is capable of
till's avatar
till committed
531
532
533
dynamically loading object files into a running program.
Cexp keeps track of module dependencies. Note that this
only covers symbol table dependencies. Cexp rejects
till's avatar
till committed
534
535
536
537
unloading a module 'A' if there is still another module,
'B' loaded which had undefined symbols resolved against 'A's
symbols. Obviously, more subtle dependencies, such as
threads using a modules text or data cannot be tracked
strauman's avatar
strauman committed
538
easily and are ignored.
till's avatar
till committed
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572

An object file (AKA 'module') can be loaded invoking the
command:

  Cexp> someModule=cexpModuleLoad("someModule.o")

The loader returns a 'module ID', which in this example
is stored in the user variable 'someModule'. If errors
occur during the load (such as undefined references or
multiple symbol definitions), they are reported and a NULL
module ID is returned.

After loading, C++ static constructors are executed and
exception handler frames are registered. Note that these
features are probably only supported on ELF and, especially
the exception handling, might only work for gcc compiled
code. Unfortunately, even different versions of gcc and/or
target architectures involve varying implementations of
exception handling - YMMV...

An (unused) module can be unloaded by passing its ID to
cexpModuleUnload() (prior to unloading, the C++ static
destructors are executed):

  Cexp> cexpModuleUnload(someModule)

Two more routines are useful in this context:

  cexpModuleInfo([ID])

prints info about a specific module (ID) or all currently
loaded modules if passed NULL ID.

  cexpModuleFindByName("regexp")
till's avatar
till committed
573

till's avatar
till committed
574
575
576
searches the list of modules for a regular expression and
reports the IDs of all matches. It returns the first ID
found (or NULL if there was no match).
strauman's avatar
strauman committed
577

till's avatar
till committed
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
Search Path for Loadable Modules
- - - - - - - - - - - - - - - - -
If set, the PATH environment variable is consulted
by the loader when it tries to locate object files
by subsequently prepending colon-separated search
paths as listed in PATH to the file name. The search
starts with the first component and stops as soon
as the file is found. An empty search path (two colons
in a row or a leading or trailing colon) is equivalent
to the current directory ('.'). Note that the current
directory is not searched if PATH is set but does not
contain the CWD. PATH is ignored if the object name
contains a (relative or absolute) directory path already.

Example: search '/tmp', '/TFTP/BOOTP_HOST/blah/bin' and
finally the CWD:

setenv("PATH","/tmp:/TFTP/BOOTP_HOST/blah/bin:",1)

strauman's avatar
strauman committed
597
598
Building loadable modules
- - - - - - - - - - - - -
599
Note the important difference between a run-time loaded/linked
strauman's avatar
strauman committed
600
object and a _shared_ object (such as a shared library). The 
601
latter, although also 'run-time loaded' is different from
strauman's avatar
strauman committed
602
603
604
605
the former. A shared object is shared among several entities
using disjunct address spaces (e.g. different UNIX processes).
Supporting shared objects involves PIC and GOTs.

606
607
608
609
***********************************************************

CEXP does _not_ support shared libraries! An attempt to run
a loaded module that was compiled with '-fpic', '-fPIC'
till's avatar
till committed
610
and/or linked with the '-shared' options will crash hard.
611
612
613
614

***********************************************************


strauman's avatar
strauman committed
615
616
617
618
619
620
621
622
To illustrate the difference, consider a LINUX system
running two instances of the 'cexp' demo. 'cexp' is linked
against glibc which is a shared library - both instances
of 'cexp' use the _same_ copy of libc residing in physical memory.
Let's now assume the both of the demo programs issue

	cexpModuleLoad("someObjectFile.o")

623
Cexp's run-time loader doesn't support shared objects - hence
strauman's avatar
strauman committed
624
both instances of 'cexp' will end up with their own copy of
625
'someObjectFile' which will get loaded twice to physical memory.
strauman's avatar
strauman committed
626
627
628

The main target of Cexp is RTEMS, a real-time OS which has
a global address space shared by all threads. Hence, there is
629
no need for shared object support but run-time loading
strauman's avatar
strauman committed
630
631
code is still very desirable.

632
The Cexp run-time loader simply accepts _any_ relocatable object file.
strauman's avatar
strauman committed
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
The simplest modules are just object files:

	cross-gcc -c -O some_object.c

Multiple objects can be combined using the '-r' linker option

	cross-ld  -o some_object.o -r some1.o some2.o some3.o

It is also possible to convert entire (static) libraries into
loadable objects:

	cross-ld  -o lib_object.o -r --whole-archive libSome.a

The 'system' symbol table is itself an object file from where
Cexp loads the initial symbol table at startup. The executable
itself may be used for that purpose (unless it is a pure binary
as it is the case on some embedded systems). Alternatively
(for saving memory and time), a stripped-down object file containing
651
652
only the symbol table may be generated using the 'objcopy'
tool, e.g:
strauman's avatar
strauman committed
653

654
	cross-objcopy --extract-symbol cexp cexp.sym
strauman's avatar
strauman committed
655
656
	cexp -s cexp.sym

657
658
659
660
661
662
663
This requires binutils 2.18 or later. Alternatively, the
'xsyms' utility that comes with CEXP may be used

	xsyms cexp cexp.sym

but this is deprecated.

664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
Weak Symbol Support
- - - - - - - - - -
Cexp can handle weak symbol references (including weak
undefined symbols). HOWEVER, this works only one-way:
If a module with a strong symbol is already loaded
(or if the base system exports a strong definition)
then loading a module with a weak reference works as
expected, i.e., the references are bound to the existing
strong definition.

OTOH, if a weak definition exists (either in the system
symbol table or a previosly loaded module) then it is
NOT possible to load another module with a strong
definition. (If you think about it: all relocations of
the already linked and loaded stuff would have to be
redone -- including the relocations within the base
executable...).

When a module with a weak undefined reference is loaded
then (provided that no existing definition is found in
the system or other modules) a new NULL symbol is created
which is otherwise treated like a newly defined COMMON
symbol.


strauman's avatar
strauman committed
689
690
Loadable Modules 'Magic'
- - - - - - - - - - - - -
691
When loading modules, 'Cexp' does some magic operations on
strauman's avatar
strauman committed
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
special symbols it recognizes. Note that the all-capital names
given here are macros - the actual names can be found in the
header.

	CEXPMOD_INITIALIZER_SYM (defined in cexpmodP.h), when
		present in a loaded module, a routine with this
		name is invoked by cexpModuleLoad() just after calling
		C++ constructors.
		This 'module-constructor' routine [for C++ information
		see below] can be used to initialize a (non-C++) module.

	CEXPMOD_FINALIZER_SYM, when present in a loaded module,
		a routine with this name is invoked just prior to calling
		C++ destructors and unloading the module.
		The FINALIZER may reject the unloading attempt by
		returning a nonzero value.

	CEXP_HELP_TAB (defined in cexpHelp.h). A module may define
		(multiple) 'help' tables (their name must begin with
		the magic CEXP_HELP_TAB string) for providing help
		information about specific symbols contained in a module.
		See cexpHelp.h for more information about the help information.
		cexpModuleLoad() automatically registers this data.

Building the 'Main' Application
- - - - - - - - - - - - - - - -

When linking a traditional application, only objects referenced by the
720
application will be linked into the executable. E.g. an application which
strauman's avatar
strauman committed
721
722
723
724
725
726
727
728
729
730
does not use 'printf()' will not have that routine available. In the
normal case, this is fine. When using a runtime loader like CEXP, there
arises a problem: imagine you want to load a piece of code which _does_
use 'printf()':

  - Cexp resolves the module's undefined symbols, encounters 'printf'
    but doesn't find it in its symbol table - it rejects loading your
    module.

  - It is not possible either to simply link your module against libc!
731
    The module might reference other libc objects which _are_ present
strauman's avatar
strauman committed
732
733
734
735
736
    in the application already - Cexp would complain that their symbols
    are already defined.

Therefore, the 'primary' or 'system' application should be built to 
include ALL parts of the core libraries (such as the C-library, the
737
RTEMS executive managers etc.) which will possibly be used by modules
strauman's avatar
strauman committed
738
739
740
741
742
743
744
745
746
747
748
749
750
loaded into the running system.

Hence, there is a new task for the system designer which (for a statically
linked application) would be automatically performed by the linker:
You must now tailor/configure the core parts of your system. This
is essentially a memory/functionality tradeoff (Sidenote: vxWorks
is configured in such a way, too). This 'tailoring' essentially
means that you have to tell the linker what parts of the basic
system libraries (RTEMS managers, CPU/BSP support, libc, networking
etc.) should forcibly be included into the link.

Note that it is only necessary to take into account the libraries
needed by CEXP and the OS itself during this process.
751
Any library CEXP does _not_ depend on, directly or indirectly, can
strauman's avatar
strauman committed
752
753
of course be dynamically loaded any time later.

754
755
756
757
758
759
760
A new tool 'ldep' is available greatly alleviating the task
of analyzing link file interdependency and generating symbol
lists etc. It is available as part of the 'GeSys' package
(www.slac.stanford.edu/~strauman/rtems/gesys) and comes with
more documentation. Some info is available here:
http://www/~strauman/rtems/epics/README.config

761
762
763
764
Using the Parser from a Program
- - - - - - - - - - - - - - - -

In some cases, you might want to access/use symbols in loaded modules from 
765
a program rather than through the Cexp shell or a shell script. If the
766
767
768
module providing the 'caller' is loaded after the module defining the 'callee'
this is transparent. E.g., let a module 'A' define a function 'a()' and module
'B' define a routine 'b()' which shall call 'a()'. If module 'A' was loaded
769
before 'B' then Cexp's linker will resolve the reference and no special
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
treatment is required.

If, however, module 'B' must for some reason be loaded before 'A' then
'B' cannot simply call 'a()' since that would result in a linker error.

Here's how you can load a module from a program and use symbols exported by
that (or any other) module (no error checking shown):

void (*p_a)();             /* declare function pointer to 'a()'              */
	cexpModuleLoad("A",0); /* use file name as module name; PATH is searched */

    /* lookup the symbol */
	p_a = cexpSymValue(cexpSymLookup("a",0));
	/* call function     */	
	if ( !p_a )
		/* error; symbol not found */
	else
		p_a();


Another method is using the interpreter (slower, of course). Since the
interpreter is re-entrant, this involves dragging around context information:

793
CexpParserCtx c = cexpCreateParserCtx(0,0,0,0);
794
795
796
797
798
799

	/* these two calls could/should be wrapped into 'cexpParseLine'... */
	cexpResetParserCtx(c, "a()");
	cexpparse(c);

Note that more documentation about these calls can be found in 'cexp.h'.
800
Note also that some of the Cexp library calls require more arguments
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
than apparent from the examples in other sections of this document where
the shell is explained!  Most routines are geared for shell use and declare
the most important arguments first, followed by optional arguments and they
use defaults when these optional arguments are NULL. The shell implicitely
sets extra arguments to zero so that when you type e.g.,

  'cexpModuleLoad("file.o")'

this effectively is expanded to 'cexpModuleLoad("file.o",NULL)' since
cexpModuleLoad() expects two parameters. OTOH, if you code
cexpModuleLoad("file.o") in a program then the compiler will warn you
and the second argument will be undefined. Therefore, if you want to use
library calls from a program you should consult 'cexp.h' for reference.

Before you can use the Cexp library, it must be properly initialized:

CEXP Liberary Initialization
- - - - - - - - - - - - - - -
When using the 'cexp_main()' AKA shell entry point, the library is initialized
automatically (you might still want to use cexpInit() if you intend to use your
own signal handler...).
822
If you want to use Cexp from a program as shown in the examples above, the library
823
824
825
826
827
must be initialized before using it:

	cexpInit(0);            /* initialize internal data; use default signal handling   */
	cexpModuleLoad(0,0);	/* attach built-in symbol table. If you dont have a built- */
	                        /* in table, you must pass a filename argument (see cexp.h)*/
828

strauman's avatar
strauman committed
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
C++ Information
- - - - - - - -

Dynamically loading C++ modules requires additional support.

	1) static constructors and destructors
	2) multiply defined instances of code (templates, multiple inclusion of headers)
	3) exception handling

Unfortunately, all of these items are highly implementation dependent and it is therefore
unlikely that C++ support works for compilers other than gcc. Alas, there are even
significant differences between different versions of gcc and target ABI's.

IMHO, C++ is unreadable, unportable, bloat-prone and should be avoided if at all
possible. That said, let's dive into the details:

	1) static constructors and destructors take care of initializing/finalizing
	   static objects such as:

		BlahClass	blahObj(a,b,c);
		int			test=testInitialize(x,y);

	   gcc creates code for initializing/finalizing these kinds of objects and
	   tags this code with symbols similar to '__GLOBAL__.I.xxx' / '__GLOBAL__.D.xxx'.
	   If Cexp encounters such symbols, it builds constructor/destructor lists when
	   loading the module and executes the respective code after loading / prior to
	   unloading.
856
857
858
	   Later versions of gcc use the '__cxa_atexit' callback mechanism which requires
	   CEXP to call '__cxa_finalize()' when a module is unloaded. This seems to be
	   a C++ ABI standard and is supported by Cexp.
strauman's avatar
strauman committed
859
	   There are other possible implementations (such as creating special ctor/dtor
860
	   sections) which are not supported - as I said, there is no general way for 
strauman's avatar
strauman committed
861
862
863
864
865
866
867
868
	   handling C++ :-(

	2) C++ allows for multiple definitions/instantiations of code (a header included
	   by more than one 'xxx.cc' file may define class members) plus there is the
	   'template' ''feature'' of C++.
	   These let C++ code size _explode_ keeping hardware manufacturers happy.
	   Gcc deals with this problem by putting every piece and bit of (possibly
	   redundant code and/or data) into a separate '.gnu.linkonce.t.xyzkljsbjj783c'
869
	   SECTION. The idea is that the linker (which is in our case 'Cexp') eliminates
strauman's avatar
strauman committed
870
871
872
873
874
875
876
877
	   redundant sections. The sad result are bloated object files and Cexp symbol
	   tables (Unfortunately, Cexp must keep _all_ of the zillions of
	   '.gnu.linkonce.x.yzu' symbols around since a redundant copy might be
	   loaded/linked at any time :-0 )

       You can help CEXP, however by using proper linker scripts when linking
       large C++ applications (such as EPICS).
       If you are sure that your large application is the only (C++) object (sharing
878
       common code), you may provide the linker with a script which instructs it
strauman's avatar
strauman committed
879
880
881
882
883
884
885
886
887
888
889
       to integrate _all_ the "gnu.linkonce.t.xxx" sections into ".text" etc.

       If you don't believe me - just let CEXP load an EPICS IOC and list all the
       'linkonce' symbols:

         lkup("linkonce")

       you will be surprised!
	   
	3) exception handling (while nice at the abstracion level) is another field
	   that is highly architecture/compiler/version dependent and a seed to bloat.
890
	   Even reentrancy might not be taken for granted.
strauman's avatar
strauman committed
891
892
893
894
895
896
897
	   Gcc uses two flavors of exception handling 'longjmp' and 'eh_frame' style
	   which are (to some extent) both supported by Cexp...

As is said: C++ IS THE DEVIL in the detail. While linking C (and even fortran etc.)
code compiled with different compilers (e.g. libraries) is seamless under a common
ABI, it is very complex if not impossible with C++.

898
899
900
901
902
903
Important PowerPC ABI Information
---------------------------------
Cexp currently does not support the short data areas according to the SYSV/EABI
specifications. Short data sections are merged by Cexp into the normal data
segment and hence cannot be accessed via R13/R2. Hence, when compiling loadable
modules, the compiler's -msdata flag must not be set to either of 'eabi', 'sysv'
904
905
906
or 'use'. If you try to load code compiled with an improper -msdata setting,
you will get a 'Relocation of unsupported size/type requested' error for the
relocation type 'R_PPC_EMB_SDA21' or 'R_PPC_SDAREL16' or similar.
907

till's avatar
till committed
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
PowerPC Branch Relocation Information
-------------------------------------
When generating PPC code, gcc usually (i.e., unless giving the -mlongcall
option which -- according to the gcc documentation -- may some day
disappear) emits branch instructions which cannot jump farther than
+/-32MB from the current PC location. This is a problem on systems with
more that 64MB of memory. Since the space required for loadable modules
is allocated from the regular 'malloc'-heap there is no guarantee
that all modules (and the base executable) are all located closely
enough together to be reachable and the loader may thus fail
complaining that a R_PPC_REL24 relocation cannot be resolved.

In order to solve this problem, Cexp now supports 'memory segments'
and tries to load all text sections into a separate segment which
is smaller than 32M and entirely reachable from the base executable.

There are different ways for configuring Cexp's PPC text segment:

 1) Do not reserve/use a dedicated text segment (OK for boards
    with less than 32M of memory).
 2) Automatically reserve memory for the text segment within the 
    base executable's '.bss' section.
 3) Application provides memory via a two global variables
 4) Application provides memory via linker script.

The Cexp '--enable-text-segment' configuration option is used
to define the desired behavior when Cexp is 'configure'd.

 1) Configure with '--enable-text-segment=no'

 2) Specify the desired amount of memory to be automatically
    reserved (in bytes):

	  --enable-text-segment=<number_gt_zero>

	e.g.

	  --enable-text-segment=0x800000

	or simply (for a default of 8MB)

	  --enable-text-segment=yes

 3) Configure with --enable-text-segment=0 (which is also
    the default, i.e, you may omit the configuration option).

	The application must provide two variables

	  unsigned long cexpTextRegionSize = <desired size>;
	  unsigned char cexpTextRegion[<desired size>];

 4) Configure with --enable-text-segment=0 (which is also
    the default, i.e, you may omit the configuration option).

    The application provides two symbols (most likely from
	a linker script) defining the start and end of the
	text segment (keep in mind that this region AND the 
	text sections of the base executable MUST be closer
	than 32MB together)

	_cexpTextRegionStart, _cexpTextRegionEnd
	
NOTE: Some BSP's linker script ('linkcmds') for no good reason
      limit the size of an executable to <4MB by defining a
	  MEMORY {} region -- often named 'CODE' -- in the linker
	  script which is only 4MB in size. 
	  When you try to reserve a reasonably sized text segment
	  on such a system (e.g., 16MB on a board with 512MB of 
	  memory) then linking the application will fail because
	  it wouldn't fit into the small 'CODE' region.
	  You can simply fix this by increasing the size of this
	  region to 32MB, e.g., the fixed MEMORY definition of the
	  powerpc/shared linkcmds would look like this:

        MEMORY {
          VECTORS : ORIGIN = 0x0 ,    LENGTH = 0x3000
             CODE : ORIGIN = 0x3000 , LENGTH = 32M - 0x3000
        }

	  (the original LENGTH of CODE was 4MB.)


strauman's avatar
strauman committed
990
991
992
LICENSING & DISCLAIMERS
-----------------------

993
Consult the separate LICENSE file for details.