Subject: v19i055: Flex, a fast LEX replacement, Part01/07 Newsgroups: comp.sources.unix Sender: sources Approved: rsalz@uunet.UU.NET Submitted-by: Vern Paxson Posting-number: Volume 19, Issue 55 Archive-name: flex2/part01 #! /bin/sh # This is a shell archive. Remove anything before this line, then unpack # it by saving it into a file and typing "sh file". To overwrite existing # files, type "sh file -c". You can also feed this as standard input via # unshar, or by typing "sh 'flex/COPYING' <<'END_OF_FILE' XFlex carries the copyright used for BSD software, slightly modified Xbecause it originated at the Lawrence Berkeley (not Livermore!) Laboratory, Xwhich operates under a contract with the Department of Energy: X X Copyright (c) 1989 The Regents of the University of California. X All rights reserved. X X This code is derived from software contributed to Berkeley by X Vern Paxson. X X The United States Government has rights in this work pursuant to X contract no. DE-AC03-76SF00098 between the United States Department of X Energy and the University of California. X X Redistribution and use in source and binary forms are permitted X provided that the above copyright notice and this paragraph are X duplicated in all such forms and that any documentation, X advertising materials, and other materials related to such X distribution and use acknowledge that the software was developed X by the University of California, Berkeley. The name of the X University may not be used to endorse or promote products derived X from this software without specific prior written permission. X THIS SOFTWARE IS PROVIDED ``AS IS'' AND WITHOUT ANY EXPRESS OR X IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED X WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. X XThis basically says "do whatever you please with this software except Xremove this notice". X XNote that the "flex.skel" scanner skeleton carries no copyright notice. XYou are free to do whatever you please with scanners generated using flex; Xfor them, you are not even bound by the above copyright. END_OF_FILE if test 1554 -ne `wc -c <'flex/COPYING'`; then echo shar: \"'flex/COPYING'\" unpacked with wrong size! fi # end of 'flex/COPYING' fi if test -f 'flex/Changes' -a "${1}" != "-c" ; then echo shar: Will not clobber existing file \"'flex/Changes'\" else echo shar: Extracting \"'flex/Changes'\" \(3897 characters\) sed "s/^X//" >'flex/Changes' <<'END_OF_FILE' XChanges between 2.1 beta-test release of June '89 and previous release: X X User-visible: X X - -p flag generates a performance report to stderr. The report X consists of comments regarding features of the scanner rules X which result in slower scanners. X X - -b flag generates backtracking information to lex.backtrack. X This is a list of scanner states which require backtracking X and the characters on which they do so. By adding rules X one can remove backtracking states. If all backtracking states X are eliminated, the generated scanner will run faster. X Backtracking is not yet documented in the manual entry. X X - Variable trailing context now works, i.e., one can have X rules like "(foo)*/[ \t]*bletch". Some trailing context X patterns still cannot be properly matched and generate X error messages. These are patterns where the ending of the X first part of the rule matches the beginning of the second X part, such as "zx*/xy*", where the 'x*' matches the 'x' at X the beginning of the trailing context. Lex won't get these X patterns right either. X X - Faster scanners. X X - End-of-file rules. The special rule "<>" indicates X actions which are to be taken when an end-of-file is X encountered and yywrap() returns non-zero (i.e., indicates X no further files to process). See manual entry for example. X X - The -r (reject used) flag is gone. flex now scans the input X for occurrences of the string "REJECT" to determine if the X action is needed. It tries to be intelligent about this but X can be fooled. One can force the presence or absence of X REJECT by adding a line in the first section of the form X "%used REJECT" or "%unused REJECT". X X - yymore() has been implemented. Similarly to REJECT, flex X detects the use of yymore(), which can be overridden using X "%used" or "%unused". X X - Patterns like "x{0,3}" now work (i.e., with lower-limit == 0). X X - Removed '\^x' for ctrl-x misfeature. X X - Added '\a' and '\v' escape sequences. X X - \ now works for octal escape sequences; previously X \0 was required. X X - Better error reporting; line numbers are associated with rules. X X - yyleng is a macro; it cannot be accessed outside of the X scanner source file. X X - yytext and yyleng should not be modified within a flex action. X X - Generated scanners #define the name FLEX_SCANNER. X X - Rules are internally separated by YY_BREAK in lex.yy.c rather X than break, to allow redefinition. X X - The macro YY_USER_ACTION can be redefined to provide an action X which is always executed prior to the matched rule's action. X X - yyrestart() is a new action which can be used to restart X the scanner after it has seen an end-of-file (a "real" one, X that is, one for which yywrap() returned non-zero). It takes X a FILE* argument indicating a new file to scan and sets X things up so that a subsequent call to yylex() will start X scanning that file. X X - Internal scanner names all preceded by "yy_" X X - lex.yy.c is deleted if errors are encountered during processing. X X - Comments may be put in the first section of the input by preceding X them with '#'. X X X X Other changes: X X - Some portability-related bugs fixed, in particular for machines X with unsigned characters or sizeof( int* ) != sizeof( int ). X Also, tweaks for VMS and Microsoft C (MS-DOS), and identifiers all X trimmed to be 31 or fewer characters. Shortened file names X for dinosaur OS's. Checks for allocating > 64K memory X on 16 bit'ers. Amiga tweaks. Compiles using gcc on a Sun-3. X X - Compressed and fast scanner skeletons merged. X X - Skeleton header files done away with. X X - Generated scanner uses prototypes and "const" for __STDC__. X X - -DSV flag is now -DSYS_V for System V compilation. X X - Removed all references to FTL language. X X - Software now covered by BSD Copyright. X X - flex will replace lex in subsequent BSD releases. END_OF_FILE if test 3897 -ne `wc -c <'flex/Changes'`; then echo shar: \"'flex/Changes'\" unpacked with wrong size! fi # end of 'flex/Changes' fi if test -f 'flex/MISC' -a "${1}" != "-c" ; then echo shar: Will not clobber existing file \"'flex/MISC'\" else echo shar: Extracting \"'flex/MISC'\" \(7247 characters\) sed "s/^X//" >'flex/MISC' <<'END_OF_FILE' XMiscellaneous flex stuff. In here you'll find an out-of-date VMS makefile Xfor flex and notes someone sent me regarding converting flex to deal with X8 bit characters. This stuff is provided so that ambitious folks can pick Xit up and turn it into something viable. I'd appreciate being sent any Xupdates resulting from work on this. X XThe makefile is from Fred Brehm (fwb@demon.siemens.com); the 8 bit chars Xfrom Earle Horton (arizona!earleh@eleazar.Dartmouth.EDU). X X X X############################ VMS MAKEFILE ############################## X X# VMS make file for "flex" tool X X# Redefine the following for your own environment XBIN = tools$$exe XLIB = tools$$library XINC = tools$$include XMAN = tools$$manual XLINKFLAGS = ,$(LIB):cc/opt X XSKELETON_FILE = "DEFAULT_SKELETON_FILE=""$(LIB):FLEX.SKEL""" XF_SKELETON_FILE = "FAST_SKELETON_FILE=""$(LIB):FLEX.FASTSKEL""" X XCCFLAGS = VMS XFLEX_FLAGS = -is X XFLEXOBJS = ccl.obj dfa.obj ecs.obj main.obj misc.obj nfa.obj parse.obj \ Xscan.obj sym.obj tblcmp.obj yylex.obj X XOBJ = ccl.obj,dfa.obj,ecs.obj,main.obj,misc.obj,nfa.obj,parse.obj,\ Xscan.obj,sym.obj,tblcmp.obj,yylex.obj X Xdefault : flex Xinstall : inc lib bin man Xinc : $(INC):flexskeldef.h $(INC):fastskeldef.h $(INC):flexskelcom.h Xlib : $(LIB):flex.skel $(LIB):flex.fastskel Xbin : $(BIN):flex X flex :== $$ $(BIN):flex Xman : $(MAN):flex.doc X X$(INC):flexskeldef.h : flexskeldef.h X copy flexskeldef.h $(INC) X$(INC):fastskeldef.h : fastskeldef.h X copy fastskeldef.h $(INC) X$(INC):flexskelcom.h : flexskelcom.h X copy flexskelcom.h $(INC) X$(LIB):flex.skel : flex.skel X copy flex.skel $(LIB) X$(LIB):flex.fastskel : flex.fastskel X copy flex.fastskel $(LIB) X$(BIN):flex.exe : flex.exe X copy flex.exe $(BIN) X$(MAN):flex.doc : flex.doc X copy flex.doc $(MAN) X Xflex : flex.exe X flex :== $$ 'f$$environment("default")'flex X Xflex.exe : $(FLEXOBJS) X link /exe=flex $(OBJ) $(LINKFLAGS) X Xparse.h : parse.c X Xparse.c : parse.y X yacc :== $$ sys$$sysroot:[shellexe]yacc X yacc -d parse.y X copy y_tab.c parse.c X copy y_tab.h parse.h X del y_tab.h;*,y_tab.c;* X X#scan.c : scan.l X# flex $(FLEX_FLAGS) scan.l X# copy lex_yy.c scan.c X# del lex_yy.c;* Xscan.c : scan.dist X copy scan.dist scan.c X Xccl.obj : ccl.c flexdef.h X cc /define=$(CCFLAGS) ccl.c Xdfa.obj : dfa.c flexdef.h X cc /define=$(CCFLAGS) dfa.c Xecs.obj : ecs.c flexdef.h X cc /define=$(CCFLAGS) ecs.c Xmain.obj : main.c flexdef.h X cc /define=($(CCFLAGS),$(SKELETON_FILE),$(F_SKELETON_FILE)) main.c Xmisc.obj : misc.c flexdef.h X cc /define=$(CCFLAGS) misc.c Xnfa.obj : nfa.c flexdef.h X cc /define=$(CCFLAGS) nfa.c Xparse.obj : parse.c flexdef.h X cc /define=$(CCFLAGS) parse.c Xscan.obj : scan.c parse.h flexdef.h X cc /define=$(CCFLAGS) scan.c Xsym.obj : sym.c flexdef.h X cc /define=$(CCFLAGS) sym.c Xtblcmp.obj : tblcmp.c flexdef.h X cc /define=$(CCFLAGS) tblcmp.c Xyylex.obj : yylex.c parse.h flexdef.h X cc /define=$(CCFLAGS) yylex.c X Xclean : X del flex.exe;* X del scan.c;* X del parse.c;* X del parse.h;* X del lex_yy.c;* X del *.obj;* X del flex*.tmp;* X del *.diff;* X del y_tab.*;* X del makefile.;* X purge/log X copy makefile.vms makefile. X Xmakefile : makefile.vms X copy makefile.vms makefile. X Xtest : flex X define tools$$lib 'f$$environment("default")' X flex $(FLEX_FLAGS) scan.l X define tools$$lib tools$$sys:[lib] X diff/out=flex.diff scan.dist lex_yy.c X type/page flex.diff X X X X######################### Stuff for 8 Bit chars ######################## X X XEarle Horton has made a version of flex run on the MacIntosh under MPW. Not Xbeing content to scan regular ascii, he deals with all 8 bits. I also have Xwanted to write VMS filters that recognize , etc. so I contacted him Xabout his work. He seems to be unable to reach you directly--the rest of Xthis is his note... X---------------------------Note from Earle------------------ X>From arizona!earleh@eleazar.Dartmouth.EDU Fri May 27 11:33:09 1988 XReceived: from DARTVAX.DARTMOUTH.EDU by megaron.arizona.edu; Fri, 27 May 88 10:55:39 MST XReceived: from eleazar.dartmouth.edu by dartvax.dartmouth.edu (5.59/3.4ROOT) X id AA17044; Fri, 27 May 88 13:53:18 EDT XReceived: by eleazar.dartmouth.edu (5.59/3.2LEAF) X id AA15906; Fri, 27 May 88 13:53:04 EDT XDate: Fri, 27 May 88 13:53:04 EDT XFrom: arizona!earleh@eleazar.Dartmouth.EDU (Earle R. Horton) XMessage-Id: <8805271753.AA15906@eleazar.dartmouth.edu> XTo: eleazar!earleh, earleh@eleazar.Dartmouth.EDU, naucse!jdc XSubject: Re: Flex and DEC multi-nationals, help! XStatus: R X XJohn, X X I have posted the diffs to comp.sources.unix, and I think that Xthey could be used to generate a VAX C version which scans DEC Xmulti-nationals with little extra effort. I tried the address which Xyou give for Vern several times, only to have it bounce back to me for Xsome mysterious reason known only to a mail daemon somewhere. I would Xindeed appreciate it if you could forward this to him. X X I have found that the full internal representation of all Xcharacters as shorts is not necessary. Rather, it is only necessary to X X a) Prevent conversion of valid characters to negative ints. I X do this with a mask, defined for the Mac and UNIX as X "#define BYTEMASK 0xFF". Any time a char is converted to an X int, AND it with the mask. Then you don't have to declare X chars as unsigned chars, which is tiresome. X X b) The routine mkeccl() uses negation of characters for a flag X to determine whether they have been processed. I used a X debugger to estimate how many times this routine is actually X called when Flex scans scan.l. Based on what this looked X like to me, I decided that it would be appropriate to have X only the routine mkeccl() use an array of shorts for its X internal processing. When mkeccl() is called, it gets a X pointer to an array of shorts using alloca(), and then reads X the characters which it is to process into this array using X the method in (a). When mkeccl() is done, the shorts are put X back into chars, and things proceed. The internal processing X done by mkeccl() is the same. X X Flex compiled with these changes correctly scans all characters Xin the Macintosh character set, to the best of my ability to test it. XIn addition, it correctly scans chars with values > 127 on a UNIX VAX! XContrary to "common knowledge", text files which exist on a UNIX Xmachine may have valid characters in the range [\0177-\0377]. An Xexample is if I want to write a program to run on a UNIX machine which Xconverts German characters to ASCII. The ess-tset (I believe that is Xthe name) can be converted to "ss" with little loss of sense in most cases. X X The only question I have about my procedure is that it looks Xexpensive to convert the ccltbl array to shorts and back every time Xmkeccl() is called. However, the routine is not called an inordinately Xlarge number of times, and this conversion is probably not more Xexpensive than a couple of calls to strcpy(), for instance. To Xdetermine whether this is the most efficient method, one would have to Xgo through the effort to convert Flex to use shorts for all internal Xrepresentation, then profile the two methods. Not my idea of a good Xtime. X X Flex is a real nice program. I couldn't find anything else wrong Xwith it. X XEarle Horton X X END_OF_FILE if test 7247 -ne `wc -c <'flex/MISC'`; then echo shar: \"'flex/MISC'\" unpacked with wrong size! fi # end of 'flex/MISC' fi if test -f 'flex/Makefile' -a "${1}" != "-c" ; then echo shar: Will not clobber existing file \"'flex/Makefile'\" else echo shar: Extracting \"'flex/Makefile'\" \(1900 characters\) sed "s/^X//" >'flex/Makefile' <<'END_OF_FILE' X# make file for "flex" tool X X# @(#) $Header: Makefile,v 2.3 89/06/20 17:27:12 vern Exp $ (LBL) X X# Porting considerations: X# X# For System V Unix machines, add -DSYS_V to CFLAGS. X# For Vax/VMS, add -DSYS_V to CFLAGS. X# For MS-DOS, add "-DMS_DOS -DSYS_V" to CFLAGS. Create \tmp if not present. X# You will also want to rename flex.skel to something with a three X# character extension, change SKELETON_FILE below appropriately, X# For Amiga, add "-DAMIGA -DSYS_V" to CFLAGS. X# X# A long time ago, flex was successfully built using Microsoft C and X# the following options: /AL, /stack:10000, -LARGE, -Ml, -Mt128, -DSYS_V X X X# the first time around use "make first_flex" X X XSKELETON_DIR = /usr/local/lib XSKELETON_FILE = flex.skel XSKELFLAGS = -DDEFAULT_SKELETON_FILE=\"$(SKELETON_DIR)/$(SKELETON_FILE)\" XCFLAGS = -O XLDFLAGS = -s X XFLEX_FLAGS = XFLEX = ./flex XCC = cc X XFLEXOBJS = \ X ccl.o \ X dfa.o \ X ecs.o \ X gen.o \ X main.o \ X misc.o \ X nfa.o \ X parse.o \ X scan.o \ X sym.o \ X tblcmp.o \ X yylex.o X XFLEX_C_SOURCES = \ X ccl.c \ X dfa.c \ X ecs.c \ X gen.c \ X main.c \ X misc.c \ X nfa.c \ X parse.c \ X scan.c \ X sym.c \ X tblcmp.c \ X yylex.c X Xflex : $(FLEXOBJS) X $(CC) $(CFLAGS) -o flex $(LDFLAGS) $(FLEXOBJS) X Xfirst_flex: X cp initscan.c scan.c X make $(MFLAGS) flex X Xparse.h parse.c : parse.y X $(YACC) -d parse.y X @mv y.tab.c parse.c X @mv y.tab.h parse.h X Xscan.c : scan.l X $(FLEX) -ist $(FLEX_FLAGS) scan.l >scan.c X Xscan.o : scan.c parse.h X Xmain.o : main.c X $(CC) $(CFLAGS) -c $(SKELFLAGS) main.c X Xflex.man : flex.1 X nroff -man flex.1 >flex.man X Xlint : $(FLEX_C_SOURCES) X lint $(FLEX_C_SOURCES) > flex.lint X Xdistrib : X mv scan.c initscan.c X chmod 444 initscan.c X $(MAKE) $(MFLAGS) clean X Xclean : X rm -f core errs flex *.o parse.c *.lint parse.h flex.man tags X Xtags : X ctags $(FLEX_C_SOURCES) X Xvms : flex.man X $(MAKE) $(MFLAGS) distrib X Xtest : X $(FLEX) -ist $(FLEX_FLAGS) scan.l | diff scan.c - END_OF_FILE if test 1900 -ne `wc -c <'flex/Makefile'`; then echo shar: \"'flex/Makefile'\" unpacked with wrong size! fi # end of 'flex/Makefile' fi if test -f 'flex/README' -a "${1}" != "-c" ; then echo shar: Will not clobber existing file \"'flex/README'\" else echo shar: Extracting \"'flex/README'\" \(2472 characters\) sed "s/^X//" >'flex/README' <<'END_OF_FILE' X// $Header: README,v 2.4 89/06/20 17:12:01 vern Exp $ X XThis is release 2.1 of flex - a beta release. X XThe flex distribution consists of the following files: X X README This message X X Makefile X flexdef.h X parse.y X scan.l X ccl.c X dfa.c X ecs.c flex sources X gen.c X main.c X misc.c X nfa.c X sym.c X tblcmp.c X yylex.c X X initscan.c pre-flex'd version of scan.l X X flex.skel skeleton for generated scanners X X flex.1 manual entry X X Changes Differences between this release and the previous one X COPYING flex's copyright X MISC miscellaneous stuff (e.g., old VMS Makefile) which X almost no one will care about X X XIf you have installed a previous version of flex, delete it (after making Xbackups, of course). This will entail removing the source directory, X/usr/include/{flexskelcom,fastskeldef,flexskeldef}.h, and X/usr/local/lib/flex.{skel,fastskel}, if that's where you put the various Xpieces. X X XDecide where you want to keep flex.skel (suggestion: /usr/local/lib) and Xmove it there. Edit "Makefile" and change the definition of SKELETON_FILE Xto reflect the full pathname of flex.skel. X XRead the "Porting considerations" note in the Makefile and make Xthe necessary changes. X X XTo make flex for the first time, use: X X make first_flex X Xwhich uses the pre-generated copy of the flex scanner (the scanner Xitself is written using flex). X XAssuming it builds successfully, you can test it using X X make test X XThe "diff" should not show any differences. X XIf you're feeling adventurous, rebuild scan.c using various Xcombinations of FLEX_FLAGS, each time trying "make test" when Xyou're done. To rebuild it, do X X rm scan.c X make FLEX_FLAGS="..." X Xwhere "..." is one of: X X -c X -ce X -cm X -cfe X -cFe X Xand testing using: X X make FLEX_FLAGS="..." test X X XFormat the manual entry using X X make flex.man X X XPlease send problems and feedback to: X X vern@{csam.lbl.gov,rtsg.ee.lbl.gov} or ucbvax!csam.lbl.gov!vern X X Vern Paxson X Real Time Systems X Bldg. 46A X Lawrence Berkeley Laboratory X 1 Cyclotron Rd. X Berkeley, CA 94720 X X (415) 486-6411 X X XI will be gone from mid-July '89 through mid-August '89. From August on, Xthe addresses are: X X vern@cs.cornell.edu (email sent to the former addresses should X continue to be forwarded for quite a while) X (if I'm unlucky, you'll have to send mail X to "paxson@cs.cornell.edu", so try that if X the first doesn't work) X X Vern Paxson X CS Department X Grad Office X 4126 Upson X Cornell University X Ithaca, NY 14853-7501 X X END_OF_FILE if test 2472 -ne `wc -c <'flex/README'`; then echo shar: \"'flex/README'\" unpacked with wrong size! fi # end of 'flex/README' fi if test -f 'flex/ccl.c' -a "${1}" != "-c" ; then echo shar: Will not clobber existing file \"'flex/ccl.c'\" else echo shar: Extracting \"'flex/ccl.c'\" \(4067 characters\) sed "s/^X//" >'flex/ccl.c' <<'END_OF_FILE' X/* ccl - routines for character classes */ X X/* X * Copyright (c) 1989 The Regents of the University of California. X * All rights reserved. X * X * This code is derived from software contributed to Berkeley by X * Vern Paxson. X * X * The United States Government has rights in this work pursuant to X * contract no. DE-AC03-76SF00098 between the United States Department of X * Energy and the University of California. X * X * Redistribution and use in source and binary forms are permitted X * provided that the above copyright notice and this paragraph are X * duplicated in all such forms and that any documentation, X * advertising materials, and other materials related to such X * distribution and use acknowledge that the software was developed X * by the University of California, Berkeley. The name of the X * University may not be used to endorse or promote products derived X * from this software without specific prior written permission. X * THIS SOFTWARE IS PROVIDED ``AS IS'' AND WITHOUT ANY EXPRESS OR X * IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED X * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. X */ X X#ifndef lint X Xstatic char copyright[] = X "@(#) Copyright (c) 1989 The Regents of the University of California.\n"; Xstatic char CR_continuation[] = "@(#) All rights reserved.\n"; X Xstatic char rcsid[] = X "@(#) $Header: ccl.c,v 2.0 89/06/20 15:49:07 vern Locked $ (LBL)"; X X#endif X X#include "flexdef.h" X X/* ccladd - add a single character to a ccl X * X * synopsis X * int cclp; X * char ch; X * ccladd( cclp, ch ); X */ X Xccladd( cclp, ch ) Xint cclp; Xchar ch; X X { X int ind, len, newpos, i; X X len = ccllen[cclp]; X ind = cclmap[cclp]; X X /* check to see if the character is already in the ccl */ X X for ( i = 0; i < len; ++i ) X if ( ccltbl[ind + i] == ch ) X return; X X newpos = ind + len; X X if ( newpos >= current_max_ccl_tbl_size ) X { X current_max_ccl_tbl_size += MAX_CCL_TBL_SIZE_INCREMENT; X X ++num_reallocs; X X ccltbl = reallocate_character_array( ccltbl, current_max_ccl_tbl_size ); X } X X ccllen[cclp] = len + 1; X ccltbl[newpos] = ch; X } X X X/* cclinit - make an empty ccl X * X * synopsis X * int cclinit(); X * new_ccl = cclinit(); X */ X Xint cclinit() X X { X if ( ++lastccl >= current_maxccls ) X { X current_maxccls += MAX_CCLS_INCREMENT; X X ++num_reallocs; X X cclmap = reallocate_integer_array( cclmap, current_maxccls ); X ccllen = reallocate_integer_array( ccllen, current_maxccls ); X cclng = reallocate_integer_array( cclng, current_maxccls ); X } X X if ( lastccl == 1 ) X /* we're making the first ccl */ X cclmap[lastccl] = 0; X X else X /* the new pointer is just past the end of the last ccl. Since X * the cclmap points to the \first/ character of a ccl, adding the X * length of the ccl to the cclmap pointer will produce a cursor X * to the first free space X */ X cclmap[lastccl] = cclmap[lastccl - 1] + ccllen[lastccl - 1]; X X ccllen[lastccl] = 0; X cclng[lastccl] = 0; /* ccl's start out life un-negated */ X X return ( lastccl ); X } X X X/* cclnegate - negate a ccl X * X * synopsis X * int cclp; X * cclnegate( ccl ); X */ X Xcclnegate( cclp ) Xint cclp; X X { X cclng[cclp] = 1; X } X X X/* list_character_set - list the members of a set of characters in CCL form X * X * synopsis X * int cset[CSIZE + 1]; X * FILE *file; X * list_character_set( cset ); X * X * writes to the given file a character-class representation of those X * characters present in the given set. A character is present if it X * has a non-zero value in the set array. X */ X Xlist_character_set( file, cset ) XFILE *file; Xint cset[]; X X { X register int i; X char *readable_form(); X X putc( '[', file ); X X for ( i = 1; i <= CSIZE; ++i ) X { X if ( cset[i] ) X { X register int start_char = i; X X putc( ' ', file ); X X fputs( readable_form( i ), file ); X X while ( ++i <= CSIZE && cset[i] ) X ; X X if ( i - 1 > start_char ) X /* this was a run */ X fprintf( file, "-%s", readable_form( i - 1 ) ); X X putc( ' ', file ); X } X } X X putc( ']', file ); X } END_OF_FILE if test 4067 -ne `wc -c <'flex/ccl.c'`; then echo shar: \"'flex/ccl.c'\" unpacked with wrong size! fi # end of 'flex/ccl.c' fi if test -f 'flex/ecs.c' -a "${1}" != "-c" ; then echo shar: Will not clobber existing file \"'flex/ecs.c'\" else echo shar: Extracting \"'flex/ecs.c'\" \(5548 characters\) sed "s/^X//" >'flex/ecs.c' <<'END_OF_FILE' X/* ecs - equivalence class routines */ X X/* X * Copyright (c) 1989 The Regents of the University of California. X * All rights reserved. X * X * This code is derived from software contributed to Berkeley by X * Vern Paxson. X * X * The United States Government has rights in this work pursuant to X * contract no. DE-AC03-76SF00098 between the United States Department of X * Energy and the University of California. X * X * Redistribution and use in source and binary forms are permitted X * provided that the above copyright notice and this paragraph are X * duplicated in all such forms and that any documentation, X * advertising materials, and other materials related to such X * distribution and use acknowledge that the software was developed X * by the University of California, Berkeley. The name of the X * University may not be used to endorse or promote products derived X * from this software without specific prior written permission. X * THIS SOFTWARE IS PROVIDED ``AS IS'' AND WITHOUT ANY EXPRESS OR X * IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED X * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. X */ X X#ifndef lint X Xstatic char copyright[] = X "@(#) Copyright (c) 1989 The Regents of the University of California.\n"; Xstatic char CR_continuation[] = "@(#) All rights reserved.\n"; X Xstatic char rcsid[] = X "@(#) $Header: ecs.c,v 2.0 89/06/20 15:49:39 vern Locked $ (LBL)"; X X#endif X X#include "flexdef.h" X X/* ccl2ecl - convert character classes to set of equivalence classes X * X * synopsis X * ccl2ecl(); X */ X Xccl2ecl() X X { X int i, ich, newlen, cclp, ccls, cclmec; X X for ( i = 1; i <= lastccl; ++i ) X { X /* we loop through each character class, and for each character X * in the class, add the character's equivalence class to the X * new "character" class we are creating. Thus when we are all X * done, character classes will really consist of collections X * of equivalence classes X */ X X newlen = 0; X cclp = cclmap[i]; X X for ( ccls = 0; ccls < ccllen[i]; ++ccls ) X { X ich = ccltbl[cclp + ccls]; X cclmec = ecgroup[ich]; X if ( cclmec > 0 ) X { X ccltbl[cclp + newlen] = cclmec; X ++newlen; X } X } X X ccllen[i] = newlen; X } X } X X X/* cre8ecs - associate equivalence class numbers with class members X * X * synopsis X * int cre8ecs(); X * number of classes = cre8ecs( fwd, bck, num ); X * X * fwd is the forward linked-list of equivalence class members. bck X * is the backward linked-list, and num is the number of class members. X * Returned is the number of classes. X */ X Xint cre8ecs( fwd, bck, num ) Xint fwd[], bck[], num; X X { X int i, j, numcl; X X numcl = 0; X X /* create equivalence class numbers. From now on, abs( bck(x) ) X * is the equivalence class number for object x. If bck(x) X * is positive, then x is the representative of its equivalence X * class. X */ X X for ( i = 1; i <= num; ++i ) X if ( bck[i] == NIL ) X { X bck[i] = ++numcl; X for ( j = fwd[i]; j != NIL; j = fwd[j] ) X bck[j] = -numcl; X } X X return ( numcl ); X } X X X/* mkeccl - update equivalence classes based on character class xtions X * X * synopsis X * char ccls[]; X * int lenccl, fwd[llsiz], bck[llsiz], llsiz; X * mkeccl( ccls, lenccl, fwd, bck, llsiz ); X * X * where ccls contains the elements of the character class, lenccl is the X * number of elements in the ccl, fwd is the forward link-list of equivalent X * characters, bck is the backward link-list, and llsiz size of the link-list X */ X Xmkeccl( ccls, lenccl, fwd, bck, llsiz ) Xchar ccls[]; Xint lenccl, fwd[], bck[], llsiz; X X { X int cclp, oldec, newec; X int cclm, i, j; X X#define PROCFLG 0x80 X X /* note that it doesn't matter whether or not the character class is X * negated. The same results will be obtained in either case. X */ X X cclp = 0; X X while ( cclp < lenccl ) X { X cclm = ccls[cclp]; X oldec = bck[cclm]; X newec = cclm; X X j = cclp + 1; X X for ( i = fwd[cclm]; i != NIL && i <= llsiz; i = fwd[i] ) X { /* look for the symbol in the character class */ X for ( ; j < lenccl && (ccls[j] <= i || (ccls[j] & PROCFLG)); ++j ) X if ( ccls[j] == i ) X { X /* we found an old companion of cclm in the ccl. X * link it into the new equivalence class and flag it as X * having been processed X */ X X bck[i] = newec; X fwd[newec] = i; X newec = i; X ccls[j] |= PROCFLG; /* set flag so we don't reprocess */ X X /* get next equivalence class member */ X /* continue 2 */ X goto next_pt; X } X X /* symbol isn't in character class. Put it in the old equivalence X * class X */ X X bck[i] = oldec; X X if ( oldec != NIL ) X fwd[oldec] = i; X X oldec = i; Xnext_pt: X ; X } X X if ( bck[cclm] != NIL || oldec != bck[cclm] ) X { X bck[cclm] = NIL; X fwd[oldec] = NIL; X } X X fwd[newec] = NIL; X X /* find next ccl member to process */ X X for ( ++cclp; (ccls[cclp] & PROCFLG) && cclp < lenccl; ++cclp ) X { X /* reset "doesn't need processing" flag */ X ccls[cclp] &= ~PROCFLG; X } X } X } X X X/* mkechar - create equivalence class for single character X * X * synopsis X * int tch, fwd[], bck[]; X * mkechar( tch, fwd, bck ); X */ X Xmkechar( tch, fwd, bck ) Xint tch, fwd[], bck[]; X X { X /* if until now the character has been a proper subset of X * an equivalence class, break it away to create a new ec X */ X X if ( fwd[tch] != NIL ) X bck[fwd[tch]] = bck[tch]; X X if ( bck[tch] != NIL ) X fwd[bck[tch]] = fwd[tch]; X X fwd[tch] = NIL; X bck[tch] = NIL; X } END_OF_FILE if test 5548 -ne `wc -c <'flex/ecs.c'`; then echo shar: \"'flex/ecs.c'\" unpacked with wrong size! fi # end of 'flex/ecs.c' fi if test -f 'flex/scan.l' -a "${1}" != "-c" ; then echo shar: Will not clobber existing file \"'flex/scan.l'\" else echo shar: Extracting \"'flex/scan.l'\" \(10686 characters\) sed "s/^X//" >'flex/scan.l' <<'END_OF_FILE' X/* scan.l - scanner for flex input */ X X/* X * Copyright (c) 1989 The Regents of the University of California. X * All rights reserved. X * X * This code is derived from software contributed to Berkeley by X * Vern Paxson. X * X * The United States Government has rights in this work pursuant to X * contract no. DE-AC03-76SF00098 between the United States Department of X * Energy and the University of California. X * X * Redistribution and use in source and binary forms are permitted X * provided that the above copyright notice and this paragraph are X * duplicated in all such forms and that any documentation, X * advertising materials, and other materials related to such X * distribution and use acknowledge that the software was developed X * by the University of California, Berkeley. The name of the X * University may not be used to endorse or promote products derived X * from this software without specific prior written permission. X * THIS SOFTWARE IS PROVIDED ``AS IS'' AND WITHOUT ANY EXPRESS OR X * IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED X * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. X */ X X%{ X#include "flexdef.h" X#include "parse.h" X X#ifndef lint Xstatic char copyright[] = X "@(#) Copyright (c) 1989 The Regents of the University of California.\n"; Xstatic char CR_continuation[] = "@(#) All rights reserved.\n"; X Xstatic char rcsid[] = X "@(#) $Header: scan.l,v 2.1 89/06/20 17:24:13 vern Exp $ (LBL)"; X#endif X X#define ACTION_ECHO fprintf( temp_action_file, "%s", yytext ) X#define MARK_END_OF_PROLOG fprintf( temp_action_file, "%%%% end of prolog\n" ); X X#undef YY_DECL X#define YY_DECL \ X int flexscan() X X#define RETURNCHAR \ X yylval = yytext[0]; \ X return ( CHAR ); X X#define RETURNNAME \ X (void) strcpy( nmstr, yytext ); \ X return ( NAME ); X X#define PUT_BACK_STRING(str, start) \ X for ( i = strlen( str ) - 1; i >= start; --i ) \ X unput(str[i]) X X#define CHECK_REJECT(str) \ X if ( all_upper( str ) ) \ X reject = true; X X#define CHECK_YYMORE(str) \ X if ( all_lower( str ) ) \ X yymore_used = true; X%} X X%x SECT2 SECT2PROLOG SECT3 CODEBLOCK PICKUPDEF SC CARETISBOL NUM QUOTE X%x FIRSTCCL CCL ACTION RECOVER BRACEERROR C_COMMENT C_COMMENT_2 ACTION_COMMENT X%x ACTION_STRING PERCENT_BRACE_ACTION USED_LIST X XWS [ \t\f]+ XOPTWS [ \t\f]* XNOT_WS [^ \t\f\n] X XNAME [a-z_][a-z_0-9-]* XNOT_NAME [^a-z_\n]+ X XSCNAME {NAME} X XESCSEQ \\([^\n]|[0-9]{1,3}) X X%% X static int bracelevel, didadef; X int i, indented_code, checking_used; X char nmdef[MAXLINE], myesc(); X X^{WS} indented_code = true; BEGIN(CODEBLOCK); X^#.*\n ++linenum; ECHO; /* treat as a comment */ X^"/*" ECHO; BEGIN(C_COMMENT); X^"%s"(tart)? return ( SCDECL ); X^"%x" return ( XSCDECL ); X^"%{".*\n { X ++linenum; X line_directive_out( stdout ); X indented_code = false; X BEGIN(CODEBLOCK); X } X X{WS} return ( WHITESPACE ); X X^"%%".* { X sectnum = 2; X line_directive_out( stdout ); X BEGIN(SECT2PROLOG); X return ( SECTEND ); X } X X^"%used" checking_used = REALLY_USED; BEGIN(USED_LIST); X^"%unused" checking_used = REALLY_NOT_USED; BEGIN(USED_LIST); X X X^"%"[^sx]" ".*\n { X fprintf( stderr, X "old-style lex command at line %d ignored:\n\t%s", X linenum, yytext ); X ++linenum; X } X X^{NAME} { X (void) strcpy( nmstr, yytext ); X didadef = false; X BEGIN(PICKUPDEF); X } X X{SCNAME} RETURNNAME; X^{OPTWS}\n ++linenum; /* allows blank lines in section 1 */ X\n ++linenum; return ( '\n' ); X. synerr( "illegal character" ); BEGIN(RECOVER); X X X"*/" ECHO; BEGIN(0); X"*/".*\n ++linenum; ECHO; BEGIN(0); X[^*\n]+ ECHO; X"*" ECHO; X\n ++linenum; ECHO; X X X^"%}".*\n ++linenum; BEGIN(0); X"reject" ECHO; CHECK_REJECT(yytext); X"yymore" ECHO; CHECK_YYMORE(yytext); X{NAME}|{NOT_NAME}|. ECHO; X\n { X ++linenum; X ECHO; X if ( indented_code ) X BEGIN(0); X } X X X{WS} /* separates name and definition */ X X{NOT_WS}.* { X (void) strcpy( nmdef, yytext ); X X for ( i = strlen( nmdef ) - 1; X i >= 0 && X nmdef[i] == ' ' || nmdef[i] == '\t'; X --i ) X ; X X nmdef[i + 1] = '\0'; X X ndinstal( nmstr, nmdef ); X didadef = true; X } X X\n { X if ( ! didadef ) X synerr( "incomplete name definition" ); X BEGIN(0); X ++linenum; X } X X.*\n ++linenum; BEGIN(0); RETURNNAME; X X X\n ++linenum; BEGIN(0); X{WS} X"reject" { X if ( all_upper( yytext ) ) X reject_really_used = checking_used; X else X synerr( "unrecognized %used/%unused construct" ); X } X"yymore" { X if ( all_lower( yytext ) ) X yymore_really_used = checking_used; X else X synerr( "unrecognized %used/%unused construct" ); X } X{NOT_WS}+ synerr( "unrecognized %used/%unused construct" ); X X X.*\n/{NOT_WS} { X ++linenum; X ACTION_ECHO; X MARK_END_OF_PROLOG; X BEGIN(SECT2); X } X X.*\n ++linenum; ACTION_ECHO; X X<> MARK_END_OF_PROLOG; yyterminate(); X X^{OPTWS}\n ++linenum; /* allow blank lines in section 2 */ X X /* this horrible mess of a rule matches indented lines which X * do not contain "/*". We need to make the distinction because X * otherwise this rule will be taken instead of the rule which X * matches the beginning of comments like this one X */ X^{WS}([^/\n]|"/"[^*\n])*("/"?)\n { X synerr( "indented code found outside of action" ); X ++linenum; X } X X"<" BEGIN(SC); return ( '<' ); X^"^" return ( '^' ); X\" BEGIN(QUOTE); return ( '"' ); X"{"/[0-9] BEGIN(NUM); return ( '{' ); X"{"[^0-9\n][^}\n]* BEGIN(BRACEERROR); X"$"/[ \t\n] return ( '$' ); X X{WS}"%{" { X bracelevel = 1; X BEGIN(PERCENT_BRACE_ACTION); X return ( '\n' ); X } X{WS}"|".*\n continued_action = true; ++linenum; return ( '\n' ); X X^{OPTWS}"/*" ACTION_ECHO; BEGIN(C_COMMENT_2); X X{WS} { X /* this rule is separate from the one below because X * otherwise we get variable trailing context, so X * we can't build the scanner using -{f,F} X */ X bracelevel = 0; X continued_action = false; X BEGIN(ACTION); X return ( '\n' ); X } X X{OPTWS}/\n { X bracelevel = 0; X continued_action = false; X BEGIN(ACTION); X return ( '\n' ); X } X X^{OPTWS}\n ++linenum; return ( '\n' ); X X"<>" return ( EOF_OP ); X X^"%%".* { X sectnum = 3; X BEGIN(SECT3); X return ( EOF ); /* to stop the parser */ X } X X"["([^\\\]\n]|{ESCSEQ})+"]" { X int cclval; X X (void) strcpy( nmstr, yytext ); X X /* check to see if we've already encountered this ccl */ X if ( (cclval = ccllookup( nmstr )) ) X { X yylval = cclval; X ++cclreuse; X return ( PREVCCL ); X } X else X { X /* we fudge a bit. We know that this ccl will X * soon be numbered as lastccl + 1 by cclinit X */ X cclinstal( nmstr, lastccl + 1 ); X X /* push back everything but the leading bracket X * so the ccl can be rescanned X */ X PUT_BACK_STRING(nmstr, 1); X X BEGIN(FIRSTCCL); X return ( '[' ); X } X } X X"{"{NAME}"}" { X register char *nmdefptr; X char *ndlookup(); X X (void) strcpy( nmstr, yytext ); X nmstr[yyleng - 1] = '\0'; /* chop trailing brace */ X X /* lookup from "nmstr + 1" to chop leading brace */ X if ( ! (nmdefptr = ndlookup( nmstr + 1 )) ) X synerr( "undefined {name}" ); X X else X { /* push back name surrounded by ()'s */ X unput(')'); X PUT_BACK_STRING(nmdefptr, 0); X unput('('); X } X } X X[/|*+?.()] return ( yytext[0] ); X. RETURNCHAR; X\n ++linenum; return ( '\n' ); X X X"," return ( ',' ); X">" BEGIN(SECT2); return ( '>' ); X">"/"^" BEGIN(CARETISBOL); return ( '>' ); X{SCNAME} RETURNNAME; X. synerr( "bad start condition name" ); X X"^" BEGIN(SECT2); return ( '^' ); X X X[^"\n] RETURNCHAR; X\" BEGIN(SECT2); return ( '"' ); X X\n { X synerr( "missing quote" ); X BEGIN(SECT2); X ++linenum; X return ( '"' ); X } X X X"^"/[^-\n] BEGIN(CCL); return ( '^' ); X"^"/- return ( '^' ); X- BEGIN(CCL); yylval = '-'; return ( CHAR ); X. BEGIN(CCL); RETURNCHAR; X X-/[^\]\n] return ( '-' ); X[^\]\n] RETURNCHAR; X"]" BEGIN(SECT2); return ( ']' ); X X X[0-9]+ { X yylval = myctoi( yytext ); X return ( NUMBER ); X } X X"," return ( ',' ); X"}" BEGIN(SECT2); return ( '}' ); X X. { X synerr( "bad character inside {}'s" ); X BEGIN(SECT2); X return ( '}' ); X } X X\n { X synerr( "missing }" ); X BEGIN(SECT2); X ++linenum; X return ( '}' ); X } X X X"}" synerr( "bad name in {}'s" ); BEGIN(SECT2); X\n synerr( "missing }" ); ++linenum; BEGIN(SECT2); X X X{OPTWS}"%}".* bracelevel = 0; X"reject" ACTION_ECHO; CHECK_REJECT(yytext); X"yymore" ACTION_ECHO; CHECK_YYMORE(yytext); X{NAME}|{NOT_NAME}|. ACTION_ECHO; X\n { X ++linenum; X ACTION_ECHO; X if ( bracelevel == 0 ) X { X fputs( "\tYY_BREAK\n", temp_action_file ); X BEGIN(SECT2); X } X } X X /* REJECT and yymore() are checked for above, in PERCENT_BRACE_ACTION */ X"{" ACTION_ECHO; ++bracelevel; X"}" ACTION_ECHO; --bracelevel; X[^a-z_{}"'/\n]+ ACTION_ECHO; X{NAME} ACTION_ECHO; X"/*" ACTION_ECHO; BEGIN(ACTION_COMMENT); X"'"([^'\\\n]|\\.)*"'" ACTION_ECHO; /* character constant */ X\" ACTION_ECHO; BEGIN(ACTION_STRING); X\n { X ++linenum; X ACTION_ECHO; X if ( bracelevel == 0 ) X { X fputs( "\tYY_BREAK\n", temp_action_file ); X BEGIN(SECT2); X } X } X. ACTION_ECHO; X X"*/" ACTION_ECHO; BEGIN(ACTION); X[^*\n]+ ACTION_ECHO; X"*" ACTION_ECHO; X\n ++linenum; ACTION_ECHO; X. ACTION_ECHO; X X"*/" ACTION_ECHO; BEGIN(SECT2); X"*/".*\n ++linenum; ACTION_ECHO; BEGIN(SECT2); X[^*\n]+ ACTION_ECHO; X"*" ACTION_ECHO; X\n ++linenum; ACTION_ECHO; X X[^"\\\n]+ ACTION_ECHO; X\\. ACTION_ECHO; X\n ++linenum; ACTION_ECHO; X\" ACTION_ECHO; BEGIN(ACTION); X. ACTION_ECHO; X X X{ESCSEQ} { X yylval = myesc( yytext ); X return ( CHAR ); X } X X{ESCSEQ} { X yylval = myesc( yytext ); X BEGIN(CCL); X return ( CHAR ); X } X X X.*(\n?) ECHO; X%% END_OF_FILE if test 10686 -ne `wc -c <'flex/scan.l'`; then echo shar: \"'flex/scan.l'\" unpacked with wrong size! fi # end of 'flex/scan.l' fi if test -f 'flex/sym.c' -a "${1}" != "-c" ; then echo shar: Will not clobber existing file \"'flex/sym.c'\" else echo shar: Extracting \"'flex/sym.c'\" \(7259 characters\) sed "s/^X//" >'flex/sym.c' <<'END_OF_FILE' X/* sym - symbol table routines */ X X/* X * Copyright (c) 1989 The Regents of the University of California. X * All rights reserved. X * X * This code is derived from software contributed to Berkeley by X * Vern Paxson. X * X * The United States Government has rights in this work pursuant to X * contract no. DE-AC03-76SF00098 between the United States Department of X * Energy and the University of California. X * X * Redistribution and use in source and binary forms are permitted X * provided that the above copyright notice and this paragraph are X * duplicated in all such forms and that any documentation, X * advertising materials, and other materials related to such X * distribution and use acknowledge that the software was developed X * by the University of California, Berkeley. The name of the X * University may not be used to endorse or promote products derived X * from this software without specific prior written permission. X * THIS SOFTWARE IS PROVIDED ``AS IS'' AND WITHOUT ANY EXPRESS OR X * IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED X * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. X */ X X#ifndef lint X Xstatic char copyright[] = X "@(#) Copyright (c) 1989 The Regents of the University of California.\n"; Xstatic char CR_continuation[] = "@(#) All rights reserved.\n"; X Xstatic char rcsid[] = X "@(#) $Header: sym.c,v 2.0 89/06/20 15:50:17 vern Locked $ (LBL)"; X X#endif X X#include "flexdef.h" X Xstruct hash_entry *ndtbl[NAME_TABLE_HASH_SIZE]; Xstruct hash_entry *sctbl[START_COND_HASH_SIZE]; Xstruct hash_entry *ccltab[CCL_HASH_SIZE]; X Xstruct hash_entry *findsym(); X X X/* addsym - add symbol and definitions to symbol table X * X * synopsis X * char sym[], *str_def; X * int int_def; X * hash_table table; X * int table_size; X * 0 / -1 = addsym( sym, def, int_def, table, table_size ); X * X * -1 is returned if the symbol already exists, and the change not made. X */ X Xint addsym( sym, str_def, int_def, table, table_size ) Xregister char sym[]; Xchar *str_def; Xint int_def; Xhash_table table; Xint table_size; X X { X int hash_val = hashfunct( sym, table_size ); X register struct hash_entry *sym_entry = table[hash_val]; X register struct hash_entry *new_entry; X register struct hash_entry *successor; X char *malloc(); X X while ( sym_entry ) X { X if ( ! strcmp( sym, sym_entry->name ) ) X { /* entry already exists */ X return ( -1 ); X } X X sym_entry = sym_entry->next; X } X X /* create new entry */ X new_entry = (struct hash_entry *) malloc( sizeof( struct hash_entry ) ); X X if ( new_entry == NULL ) X flexfatal( "symbol table memory allocation failed" ); X X if ( (successor = table[hash_val]) ) X { X new_entry->next = successor; X successor->prev = new_entry; X } X else X new_entry->next = NULL; X X new_entry->prev = NULL; X new_entry->name = sym; X new_entry->str_val = str_def; X new_entry->int_val = int_def; X X table[hash_val] = new_entry; X X return ( 0 ); X } X X X/* cclinstal - save the text of a character class X * X * synopsis X * char ccltxt[]; X * int cclnum; X * cclinstal( ccltxt, cclnum ); X */ X Xcclinstal( ccltxt, cclnum ) Xchar ccltxt[]; Xint cclnum; X X { X /* we don't bother checking the return status because we are not called X * unless the symbol is new X */ X char *copy_string(); X X (void) addsym( copy_string( ccltxt ), (char *) 0, cclnum, X ccltab, CCL_HASH_SIZE ); X } X X X/* ccllookup - lookup the number associated with character class text X * X * synopsis X * char ccltxt[]; X * int ccllookup, cclval; X * cclval/0 = ccllookup( ccltxt ); X */ X Xint ccllookup( ccltxt ) Xchar ccltxt[]; X X { X return ( findsym( ccltxt, ccltab, CCL_HASH_SIZE )->int_val ); X } X X X/* findsym - find symbol in symbol table X * X * synopsis X * char sym[]; X * hash_table table; X * int table_size; X * struct hash_entry *sym_entry, *findsym(); X * sym_entry = findsym( sym, table, table_size ); X */ X Xstruct hash_entry *findsym( sym, table, table_size ) Xregister char sym[]; Xhash_table table; Xint table_size; X X { X register struct hash_entry *sym_entry = table[hashfunct( sym, table_size )]; X static struct hash_entry empty_entry = X { X (struct hash_entry *) 0, (struct hash_entry *) 0, NULL, NULL, 0, X } ; X X while ( sym_entry ) X { X if ( ! strcmp( sym, sym_entry->name ) ) X return ( sym_entry ); X sym_entry = sym_entry->next; X } X X return ( &empty_entry ); X } X X X/* hashfunct - compute the hash value for "str" and hash size "hash_size" X * X * synopsis X * char str[]; X * int hash_size, hash_val; X * hash_val = hashfunct( str, hash_size ); X */ X Xint hashfunct( str, hash_size ) Xregister char str[]; Xint hash_size; X X { X register int hashval; X register int locstr; X X hashval = 0; X locstr = 0; X X while ( str[locstr] ) X hashval = ((hashval << 1) + str[locstr++]) % hash_size; X X return ( hashval ); X } X X X/* ndinstal - install a name definition X * X * synopsis X * char nd[], def[]; X * ndinstal( nd, def ); X */ X Xndinstal( nd, def ) Xchar nd[], def[]; X X { X char *copy_string(); X X if ( addsym( copy_string( nd ), copy_string( def ), 0, X ndtbl, NAME_TABLE_HASH_SIZE ) ) X synerr( "name defined twice" ); X } X X X/* ndlookup - lookup a name definition X * X * synopsis X * char nd[], *def; X * char *ndlookup(); X * def/NULL = ndlookup( nd ); X */ X Xchar *ndlookup( nd ) Xchar nd[]; X X { X return ( findsym( nd, ndtbl, NAME_TABLE_HASH_SIZE )->str_val ); X } X X X/* scinstal - make a start condition X * X * synopsis X * char str[]; X * int xcluflg; X * scinstal( str, xcluflg ); X * X * NOTE X * the start condition is Exclusive if xcluflg is true X */ X Xscinstal( str, xcluflg ) Xchar str[]; Xint xcluflg; X X { X char *copy_string(); X X /* bit of a hack. We know how the default start-condition is X * declared, and don't put out a define for it, because it X * would come out as "#define 0 1" X */ X /* actually, this is no longer the case. The default start-condition X * is now called "INITIAL". But we keep the following for the sake X * of future robustness. X */ X X if ( strcmp( str, "0" ) ) X printf( "#define %s %d\n", str, lastsc ); X X if ( ++lastsc >= current_max_scs ) X { X current_max_scs += MAX_SCS_INCREMENT; X X ++num_reallocs; X X scset = reallocate_integer_array( scset, current_max_scs ); X scbol = reallocate_integer_array( scbol, current_max_scs ); X scxclu = reallocate_integer_array( scxclu, current_max_scs ); X sceof = reallocate_integer_array( sceof, current_max_scs ); X scname = reallocate_char_ptr_array( scname, current_max_scs ); X actvsc = reallocate_integer_array( actvsc, current_max_scs ); X } X X scname[lastsc] = copy_string( str ); X X if ( addsym( scname[lastsc], (char *) 0, lastsc, X sctbl, START_COND_HASH_SIZE ) ) X lerrsf( "start condition %s declared twice", str ); X X scset[lastsc] = mkstate( SYM_EPSILON ); X scbol[lastsc] = mkstate( SYM_EPSILON ); X scxclu[lastsc] = xcluflg; X sceof[lastsc] = false; X } X X X/* sclookup - lookup the number associated with a start condition X * X * synopsis X * char str[], scnum; X * int sclookup; X * scnum/0 = sclookup( str ); X */ X Xint sclookup( str ) Xchar str[]; X X { X return ( findsym( str, sctbl, START_COND_HASH_SIZE )->int_val ); X } END_OF_FILE if test 7259 -ne `wc -c <'flex/sym.c'`; then echo shar: \"'flex/sym.c'\" unpacked with wrong size! fi # end of 'flex/sym.c' fi if test -f 'flex/yylex.c' -a "${1}" != "-c" ; then echo shar: Will not clobber existing file \"'flex/yylex.c'\" else echo shar: Extracting \"'flex/yylex.c'\" \(4412 characters\) sed "s/^X//" >'flex/yylex.c' <<'END_OF_FILE' X/* yylex - scanner front-end for flex */ X X/* X * Copyright (c) 1989 The Regents of the University of California. X * All rights reserved. X * X * This code is derived from software contributed to Berkeley by X * Vern Paxson. X * X * The United States Government has rights in this work pursuant to X * contract no. DE-AC03-76SF00098 between the United States Department of X * Energy and the University of California. X * X * Redistribution and use in source and binary forms are permitted X * provided that the above copyright notice and this paragraph are X * duplicated in all such forms and that any documentation, X * advertising materials, and other materials related to such X * distribution and use acknowledge that the software was developed X * by the University of California, Berkeley. The name of the X * University may not be used to endorse or promote products derived X * from this software without specific prior written permission. X * THIS SOFTWARE IS PROVIDED ``AS IS'' AND WITHOUT ANY EXPRESS OR X * IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED X * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. X */ X X#ifndef lint X Xstatic char copyright[] = X "@(#) Copyright (c) 1989 The Regents of the University of California.\n"; Xstatic char CR_continuation[] = "@(#) All rights reserved.\n"; X Xstatic char rcsid[] = X "@(#) $Header: yylex.c,v 2.0 89/06/20 15:50:28 vern Locked $ (LBL)"; X X#endif X X#include "flexdef.h" X#include "parse.h" X X/* yylex - scan for a regular expression token X * X * synopsis X * X * token = yylex(); X * X * token - return token found X */ X Xint yylex() X X { X int toktype; X static int beglin = false; X X if ( eofseen ) X toktype = EOF; X else X toktype = flexscan(); X X if ( toktype == EOF ) X { X eofseen = 1; X X if ( sectnum == 1 ) X { X synerr( "unexpected EOF" ); X sectnum = 2; X toktype = SECTEND; X } X X else if ( sectnum == 2 ) X { X sectnum = 3; X toktype = SECTEND; X } X X else X toktype = 0; X } X X if ( trace ) X { X if ( beglin ) X { X fprintf( stderr, "%d\t", num_rules + 1 ); X beglin = 0; X } X X switch ( toktype ) X { X case '<': X case '>': X case '^': X case '$': X case '"': X case '[': X case ']': X case '{': X case '}': X case '|': X case '(': X case ')': X case '-': X case '/': X case '\\': X case '?': X case '.': X case '*': X case '+': X case ',': X (void) putc( toktype, stderr ); X break; X X case '\n': X (void) putc( '\n', stderr ); X X if ( sectnum == 2 ) X beglin = 1; X X break; X X case SCDECL: X fputs( "%s", stderr ); X break; X X case XSCDECL: X fputs( "%x", stderr ); X break; X X case WHITESPACE: X (void) putc( ' ', stderr ); X break; X X case SECTEND: X fputs( "%%\n", stderr ); X X /* we set beglin to be true so we'll start X * writing out numbers as we echo rules. flexscan() has X * already assigned sectnum X */ X X if ( sectnum == 2 ) X beglin = 1; X X break; X X case NAME: X fprintf( stderr, "'%s'", nmstr ); X break; X X case CHAR: X switch ( yylval ) X { X case '<': X case '>': X case '^': X case '$': X case '"': X case '[': X case ']': X case '{': X case '}': X case '|': X case '(': X case ')': X case '-': X case '/': X case '\\': X case '?': X case '.': X case '*': X case '+': X case ',': X fprintf( stderr, "\\%c", yylval ); X break; X X case 1: X case 2: X case 3: X case 4: X case 5: X case 6: X case 7: X case 8: X case 9: X case 10: X case 11: X case 12: X case 13: X case 14: X case 15: X case 16: X case 17: X case 18: X case 19: X case 20: X case 21: X case 22: X case 23: X case 24: X case 25: X case 26: X case 27: X case 28: X case 29: X case 30: X case 31: X case 127: X fprintf( stderr, "\\%.3o", yylval ); X break; X X default: X (void) putc( yylval, stderr ); X break; X } X X break; X X case NUMBER: X fprintf( stderr, "%d", yylval ); X break; X X case PREVCCL: X fprintf( stderr, "[%d]", yylval ); X break; X X case 0: X fprintf( stderr, "End Marker" ); X break; X X default: X fprintf( stderr, "*Something Weird* - tok: %d val: %d\n", X toktype, yylval ); X break; X } X } X X return ( toktype ); X } END_OF_FILE if test 4412 -ne `wc -c <'flex/yylex.c'`; then echo shar: \"'flex/yylex.c'\" unpacked with wrong size! fi # end of 'flex/yylex.c' fi echo shar: End of archive 1 \(of 7\). cp /dev/null ark1isdone MISSING="" for I in 1 2 3 4 5 6 7 ; do if test ! -f ark${I}isdone ; then MISSING="${MISSING} ${I}" fi done if test "${MISSING}" = "" ; then echo You have unpacked all 7 archives. rm -f ark[1-9]isdone else echo You still need to unpack the following archives: echo " " ${MISSING} fi ## End of shell archive. exit 0