Legacy Jacket in PHP
ZOIS Technical Note TN-2009-03-01.
Author and Audience
This TN should be of interest to folk who are
intending to Jacket existing functionality found in a conventional
Transaction Processing Monitor[tp] exploiting
Apache's[ap] scaling abilities.
Written by Martin Sullivan[au],
ZOIS Limited, Cockermouth.
Abstract
A mechanism is presented to quickly jacket existing functionality
written in a programming language with a 'C' binding (if not 'C'
itself). This jacket can be used with a data-representation structure
such as JSON, or rather more plainly. It allows an 'open' escape route
for code that is currently exploiting the scaling properties of
a traditional TPM.
Introduction
Echoing work done in 1990-2001 on Java (undocumented here, but basically CORBA based Tuxedo integration) and in Perl/Mason (2002 or thereabouts and again not documented here, as yet) some experiments were made using PHP and a native interface to allow 'legacy' code, perhaps written in COBOL, C, or other such language to be run under a PHP run-time. The goal being to give potential customers an escape route from a non-transactional Transaction Processing Monitor (TPM) scaling solution.
It has been the authors experience that a number of large-scale systems have been written that use a Transaction Processing Monitor purely to generate a measure of scaling, allowing large numbers of potentially similar bits of business logic to run in a fast controlled way. Such systems pay scant regard to Transaction Management[2p] per-se and are thus ripe for transfering to an Apache based platform. Apache in itself has had to solve some major scaling[sc] issues as it has evolved and presents a readily expoitable solution.
Apache on in its own does not support two-phase commits or distributed hetrogeneous resource-based exception handling.
The goal of this work is to encapsulate the business logic of a
Tuxedo or CICS Service in a separate Shared Object library (an '.so')
which can then be invoked by a PHP based function. Data would flow
between the recycled business logic and a function, notionally called
'apply' which would then provide a regular PHP interface.
Materials and Platform
This experimental platform used Linux[lx], Apache
and PHP (including PHP CLI)[ph]. It was built using
the standard Linux 'C' compiler, gcc, and Emacs[em]
provided the majority of the programming environment. To write the PHP
extension (called 'apply', see below) then a PHP development
environment needs to be used. On Ubuntu Linux this package is known as
php5-dev and can be downloaded and installed using the Synaptic package
manager.
Method
In this scheme mod_php and Apache would be used to provide the work distribution and so forth, allowing a presentation layer to be written in PHP relatively quickly. It was considered that any existing presentation layer would likely be junked at this stage. The existing back-end legacy code would be encapsulated in C allowing a translation from a proprietary interface into a simple string based one either using a private representation or something like XML or JSON. The business logic written in COBOL under CICS as an example, would then exist in a separate shared library and be invoked by a standardised interface. With such an interface a remote exception extension could be built in due course. At time of writing, transaction management would be an obvious extension, but as yet there is no code to accomplish this. The PHP example would then be like this:
$packed_output = apply ("OLDTHING", json_encode ($complicated_input));
$complicated_output = json_decode ($packed_output);
The 'apply' name does not normally seemed to be used in normal PHP
environments. Its name is therefore elequently suggested by the
authors ancient LISP knowledge. Some code based on a 'hello world'
example which has been built on Zend extension-hacking tutorial[zt]. The source for apply thus becomes:
#ifdef HAVE_CONFIG_H
#include "config.h"
#endif
#include "php.h"
#include "php_apply.h"
static function_entry apply_functions[] = {
PHP_FE(apply, NULL)
{NULL, NULL, NULL}
}; /* apply_functions */
zend_module_entry apply_module_entry = {
#if ZEND_MODULE_API_NO >= 20010901
STANDARD_MODULE_HEADER,
#endif
PHP_APPLY_EXTNAME,
apply_functions,
NULL,
NULL,
NULL,
NULL,
NULL,
#if ZEND_MODULE_API_NO >= 20010901
PHP_APPLY_VERSION,
#endif
STANDARD_MODULE_PROPERTIES
}; /* apply_module_entry */
#ifdef COMPILE_DL_APPLY
ZEND_GET_MODULE(apply)
#endif
PHP_FUNCTION(apply)
{
char *name;
int name_len;
char *args;
int args_len;
if (zend_parse_parameters(ZEND_NUM_ARGS() TSRMLS_CC, "ss",
&name, &name_len,
&args, &args_len) == FAILURE)
RETURN_NULL();
static void *handle = NULL;
char *(*f) (char *);
char *error;
if (!handle)
handle = dlopen ("./simpapp.so", RTLD_NOW);
if (!handle) {
php_printf ("dlopen failed to open simpapp.so\n");
RETURN_NULL();
} /* if */
dlerror ();
*(char **) (&f) = dlsym (handle, name);
if ((error = dlerror ()) != NULL) {
php_printf ("%s\n", error);
RETURN_NULL();
} /* if */
RETURN_STRING((*f) (args), 1);
} /* PHP_FUNCTION(apply) */
The use of dlsym(3) to call a function in a symbolic library all jacketed in a PHP function seems obvious (but at time of writing was relatively obscure to searching). While in a prodution environment the Shared Object file might contain a number of public service like functions in this example proof-of-concent it contains a simpapp[sa] like function, familier to Tuxedo programmers. The make(1) file for this is:
srcdir = /u/sullivan/php
builddir = /u/sullivan/php
top_srcdir = /u/sullivan/php
top_builddir = /u/sullivan/php
EGREP = /bin/grep -E
SED = /bin/sed
CONFIGURE_COMMAND = './configure' '--enable-apply'
CONFIGURE_OPTIONS = '--enable-apply'
SHLIB_SUFFIX_NAME = so
SHLIB_DL_SUFFIX_NAME = so
RE2C = exit 0;
AWK = nawk
shared_objects_apply = apply.lo
PHP_PECL_EXTENSION = apply
PHP_MODULES = $(phplibdir)/apply.la
all_targets = $(PHP_MODULES)
install_targets = install-modules install-headers
prefix = /usr
exec_prefix = $(prefix)
libdir = ${exec_prefix}/lib
prefix = /usr
phplibdir = /u/sullivan/php/modules
phpincludedir = /usr/include/php5
CC = gcc
CFLAGS = -g -O2
CFLAGS_CLEAN = $(CFLAGS)
CPP = gcc -E
CPPFLAGS = -DHAVE_CONFIG_H
CXX =
CXXFLAGS =
CXXFLAGS_CLEAN = $(CXXFLAGS)
EXTENSION_DIR = /usr/lib/php5/20060613+lfs
PHP_EXECUTABLE = /usr/bin/php
EXTRA_LDFLAGS =
EXTRA_LIBS =
INCLUDES = -I/usr/include/php5 -I/usr/include/php5/main -I/usr/include/php5/TSRM -I/usr/include/php5/Zend -I/usr/include/php5/ext -I/usr/include/php5/ext/date/lib -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64
LFLAGS =
LDFLAGS =
SHARED_LIBTOOL =
LIBTOOL = $(SHELL) $(top_builddir)/libtool
SHELL = /bin/bash
INSTALL_HEADERS =
mkinstalldirs = $(top_srcdir)/build/shtool mkdir -p
INSTALL = $(top_srcdir)/build/shtool install -c
INSTALL_DATA = $(INSTALL) -m 644
DEFS = -DPHP_ATOM_INC -I$(top_builddir)/include -I$(top_builddir)/main -I$(top_srcdir)
COMMON_FLAGS = $(DEFS) $(INCLUDES) $(EXTRA_INCLUDES) $(CPPFLAGS) $(PHP_FRAMEWORKPATH)
all: $(all_targets)
@echo
@echo "Build complete."
@echo "Don't forget to run 'make test'."
@echo
build-modules: $(PHP_MODULES)
libphp$(PHP_MAJOR_VERSION).la: $(PHP_GLOBAL_OBJS) $(PHP_SAPI_OBJS)
$(LIBTOOL) --mode=link $(CC) $(CFLAGS) $(EXTRA_CFLAGS) -rpath $(phptempdir) $(EXTRA_LDFLAGS) $(LDFLAGS) $(PHP_RPATHS) $(PHP_GLOBAL_OBJS) $(PHP_SAPI_OBJS) $(EXTRA_LIBS) $(ZEND_EXTRA_LIBS) -o $@
-@$(LIBTOOL) --silent --mode=install cp $@ $(phptempdir)/$@ >/dev/null 2>&1
libs/libphp$(PHP_MAJOR_VERSION).bundle: $(PHP_GLOBAL_OBJS) $(PHP_SAPI_OBJS)
$(CC) $(MH_BUNDLE_FLAGS) $(CFLAGS_CLEAN) $(EXTRA_CFLAGS) $(LDFLAGS) $(EXTRA_LDFLAGS) $(PHP_GLOBAL_OBJS:.lo=.o) $(PHP_SAPI_OBJS:.lo=.o) $(PHP_FRAMEWORKS) $(EXTRA_LIBS) $(ZEND_EXTRA_LIBS) -o $@ && cp $@ libs/libphp$(PHP_MAJOR_VERSION).so
install: $(all_targets) $(install_targets)
install-sapi: $(OVERALL_TARGET)
@echo "Installing PHP SAPI module: $(PHP_SAPI)"
-@$(mkinstalldirs) $(INSTALL_ROOT)$(bindir)
-@if test ! -r $(phptempdir)/libphp$(PHP_MAJOR_VERSION).$(SHLIB_DL_SUFFIX_NAME); then \
for i in 0.0.0 0.0 0; do \
if test -r $(phptempdir)/libphp$(PHP_MAJOR_VERSION).$(SHLIB_DL_SUFFIX_NAME).$$i; then \
$(LN_S) $(phptempdir)/libphp$(PHP_MAJOR_VERSION).$(SHLIB_DL_SUFFIX_NAME).$$i $(phptempdir)/libphp$(PHP_MAJOR_VERSION).$(SHLIB_DL_SUFFIX_NAME); \
break; \
fi; \
done; \
fi
@$(INSTALL_IT)
install-modules: build-modules
@test -d modules && \
$(mkinstalldirs) $(INSTALL_ROOT)$(EXTENSION_DIR)
@echo "Installing shared extensions: $(INSTALL_ROOT)$(EXTENSION_DIR)/"
@rm -f modules/*.la >/dev/null 2>&1
@$(INSTALL) modules/* $(INSTALL_ROOT)$(EXTENSION_DIR)
install-headers:
-@if test "$(INSTALL_HEADERS)"; then \
for i in `echo $(INSTALL_HEADERS)`; do \
i=`$(top_srcdir)/build/shtool path -d $$i`; \
paths="$$paths $(INSTALL_ROOT)$(phpincludedir)/$$i"; \
done; \
$(mkinstalldirs) $$paths && \
echo "Installing header files: $(INSTALL_ROOT)$(phpincludedir)/" && \
for i in `echo $(INSTALL_HEADERS)`; do \
if test "$(PHP_PECL_EXTENSION)"; then \
src=`echo $$i | $(SED) -e "s#ext/$(PHP_PECL_EXTENSION)/##g"`; \
else \
src=$$i; \
fi; \
if test -f "$(top_srcdir)/$$src"; then \
$(INSTALL_DATA) $(top_srcdir)/$$src $(INSTALL_ROOT)$(phpincludedir)/$$i; \
elif test -f "$(top_builddir)/$$src"; then \
$(INSTALL_DATA) $(top_builddir)/$$src $(INSTALL_ROOT)$(phpincludedir)/$$i; \
else \
(cd $(top_srcdir)/$$src && $(INSTALL_DATA) *.h $(INSTALL_ROOT)$(phpincludedir)/$$i; \
cd $(top_builddir)/$$src && $(INSTALL_DATA) *.h $(INSTALL_ROOT)$(phpincludedir)/$$i) 2>/dev/null || true; \
fi \
done; \
fi
PHP_TEST_SETTINGS = -d 'open_basedir=' -d 'output_buffering=0' -d 'memory_limit=-1'
PHP_TEST_SHARED_EXTENSIONS = ` \
if test "x$(PHP_MODULES)" != "x"; then \
for i in $(PHP_MODULES)""; do \
. $$i; $(top_srcdir)/build/shtool echo -n -- " -d extension=$$dlname"; \
done; \
fi`
test: all
-@if test ! -z "$(PHP_EXECUTABLE)" && test -x "$(PHP_EXECUTABLE)"; then \
TEST_PHP_EXECUTABLE=$(PHP_EXECUTABLE) \
TEST_PHP_SRCDIR=$(top_srcdir) \
CC="$(CC)" \
$(PHP_EXECUTABLE) $(PHP_TEST_SETTINGS) $(top_srcdir)/run-tests.php -U -d extension_dir=modules/ $(PHP_TEST_SHARED_EXTENSIONS) tests/; \
elif test ! -z "$(SAPI_CLI_PATH)" && test -x "$(SAPI_CLI_PATH)"; then \
INI_FILE=`$(top_builddir)/$(SAPI_CLI_PATH) -r 'echo php_ini_loaded_file();'`; \
if test "$$INI_FILE"; then \
$(EGREP) -v '^extension[\t\ ]*=' "$$INI_FILE" > $(top_builddir)/tmp-php.ini; \
else \
echo > $(top_builddir)/tmp-php.ini; \
fi; \
TEST_PHP_EXECUTABLE=$(top_builddir)/$(SAPI_CLI_PATH) \
TEST_PHP_SRCDIR=$(top_srcdir) \
CC="$(CC)" \
$(top_builddir)/$(SAPI_CLI_PATH) $(PHP_TEST_SETTINGS) $(top_srcdir)/run-tests.php -c $(top_builddir)/tmp-php.ini -U -d extension_dir=$(top_builddir)/modules/ $(PHP_TEST_SHARED_EXTENSIONS) $(TESTS); \
else \
echo "ERROR: Cannot run tests without CLI sapi."; \
fi
utest: all
-@if test ! -z "$(SAPI_CLI_PATH)" && test -x "$(SAPI_CLI_PATH)"; then \
INI_FILE=`$(top_builddir)/$(SAPI_CLI_PATH) -r 'echo php_ini_loaded_file();'`; \
if test "$$INI_FILE"; then \
$(EGREP) -v '^extension[\t\ ]*=' "$$INI_FILE" > $(top_builddir)/tmp-php.ini; \
else \
echo > $(top_builddir)/tmp-php.ini; \
fi; \
TEST_PHP_EXECUTABLE=$(top_builddir)/$(SAPI_CLI_PATH) \
TEST_PHP_SRCDIR=$(top_srcdir) \
CC="$(CC)" \
$(top_builddir)/$(SAPI_CLI_PATH) $(PHP_TEST_SETTINGS) $(top_srcdir)/run-tests.php -c $(top_builddir)/tmp-php.ini -u -d extension_dir=$(top_builddir)/modules/ $(PHP_TEST_SHARED_EXTENSIONS) $(TESTS); \
else \
echo "ERROR: Cannot run tests without CLI sapi."; \
fi
ntest: all
-@if test ! -z "$(SAPI_CLI_PATH)" && test -x "$(SAPI_CLI_PATH)"; then \
INI_FILE=`$(top_builddir)/$(SAPI_CLI_PATH) -r 'echo php_ini_loaded_file();'`; \
if test "$$INI_FILE"; then \
$(EGREP) -v '^extension[\t\ ]*=' "$$INI_FILE" > $(top_builddir)/tmp-php.ini; \
else \
echo > $(top_builddir)/tmp-php.ini; \
fi; \
TEST_PHP_EXECUTABLE=$(top_builddir)/$(SAPI_CLI_PATH) \
TEST_PHP_SRCDIR=$(top_srcdir) \
CC="$(CC)" \
$(top_builddir)/$(SAPI_CLI_PATH) $(PHP_TEST_SETTINGS) $(top_srcdir)/run-tests.php -c $(top_builddir)/tmp-php.ini -N -d extension_dir=$(top_builddir)/modules/ $(PHP_TEST_SHARED_EXTENSIONS) $(TESTS); \
else \
echo "ERROR: Cannot run tests without CLI sapi."; \
fi
clean:
find . -name \*.gcno -o -name \*.gcda | xargs rm -f
find . -name \*.lo -o -name \*.o | xargs rm -f
find . -name \*.la -o -name \*.a | xargs rm -f
find . -name \*.so | xargs rm -f
find . -name .libs -a -type d|xargs rm -rf
rm -f libphp$(PHP_MAJOR_VERSION).la $(SAPI_CLI_PATH) $(OVERALL_TARGET) modules/* libs/*
distclean: clean
rm -f config.cache config.log config.status Makefile.objects Makefile.fragments libtool main/php_config.h stamp-h php5.spec sapi/apache/libphp$(PHP_MAJOR_VERSION).module buildmk.stamp
$(EGREP) define'.*include/php' $(top_srcdir)/configure | $(SED) 's/.*>//'|xargs rm -f
find . -name Makefile | xargs rm -f
.PHONY: all clean install distclean test
.NOEXPORT:
apply.lo: /u/sullivan/php/apply.c
$(LIBTOOL) --mode=compile $(CC) -I. -I/u/sullivan/php $(COMMON_FLAGS) $(CFLAGS_CLEAN) $(EXTRA_CFLAGS) -c /u/sullivan/php/apply.c -o apply.lo
$(phplibdir)/apply.la: ./apply.la
$(LIBTOOL) --mode=install cp ./apply.la $(phplibdir)
./apply.la: $(shared_objects_apply) $(APPLY_SHARED_DEPENDENCIES)
$(LIBTOOL) --mode=link $(CC) $(COMMON_FLAGS) $(CFLAGS_CLEAN) $(EXTRA_CFLAGS) $(LDFLAGS) -o $@ -export-dynamic -avoid-version -prefer-pic -module -rpath $(phplibdir) $(EXTRA_LDFLAGS) $(shared_objects_apply) $(APPLY_SHARED_LIBADD)
And the associated Makefile.objects:
apply.lo: /u/sullivan/php/apply.c $(LIBTOOL) --mode=compile $(CC) -I. -I/u/sullivan/php $(COMMON_FLAGS) $(CFLAGS_CLEAN) $(EXTRA_CFLAGS) -c /u/sullivan/php/apply.c -o apply.lo $(phplibdir)/apply.la: ./apply.la $(LIBTOOL) --mode=install cp ./apply.la $(phplibdir) ./apply.la: $(shared_objects_apply) $(APPLY_SHARED_DEPENDENCIES) $(LIBTOOL) --mode=link $(CC) $(COMMON_FLAGS) $(CFLAGS_CLEAN) $(EXTRA_CFLAGS) $(LDFLAGS) -o $@ -export-dynamic -avoid-version -prefer-pic -module -rpath $(phplibdir) $(EXTRA_LDFLAGS) $(shared_objects_apply) $(APPLY_SHARED_LIBADD)
It is suggested that the code be examined in conjunction with the ZEND extension documentation[zt], their efforts in explaining this stuff are undoubtedly superior to anything produced here.
It has been found that once the extension module has been compiled it needed to be installed (the generated makefile did this). Currently it calls another function supplied by an external module (in the source directory, 'simpapp', almost inevitably). Some experiments were then undertaken to assess the way that the JSON representation would work. JSON-C was downloaded, installed and experimented with. It was found that it had a difficult interface that could be liable to memory leaks if extreme care was not taken with de-referencing the 'json_object' components once work with them had been completed. This work, inwhich structure is to be added to the data involved in any interactions will be documented in a future Technical Note.
After some deliberation it was also decided to experiment with CSV as an intermediate string based mechamism. In this instance PHP provides the built-ins for some of this, but not directly, so some coding would be required ...
$packed_output = apply ("OLDTHING", str_putcsv ($complicated_input));
$complicated_output = str_getcsv ($packed_output);
PHP 5 has fputcsv, fgetcsv (for I/O) and
str_getcsv, but no str_putcsv. This omission
has spawned some lively debate and several workable examples, as well
as several dumb file-I/O based ones[gc]. C has
libcsv[cc].
These things considered, it may be easier to agree a 'private' format for the intermediate strings and use that.
In all, some coding is required to wrap the posibly private interfaces to the Transaction Program or Service in a form which is considered appropriate for the project. The experimental 'simpapp' code (which used toupper(3) to provide a case-changing service familier to generations of Tuxedo programmers) used just such a private interface based on null terminated character arrays. The the public function uppercase could be called like this:
$output_string = apply ("UPPERCASE", $input_string));
echo $output_string;
Discussion
There is much here which indicates further work. JSON functions which emulate Tuxedo's FML handling routines are obviously suggested as well as some mechanism to opequely transmit exception information. When this work has been completed it will be made the subject to future Technical Notes.
Having a reasonable understanding to the techniques (and
remembering the end-user enthusiasm for Mason/Inline::C experiments)
it was decided to write-up and leave this stuff. So, thank you for
your interest and getting this far.
References
- [tp]. Open Online Transaction Processing:
- http://www.zois.co.uk/summary.html
- [ap]. Apache:
- http://httpd.apache.org
- [au]. Martin Sullivan:
- http://www.zois.co.uk/people/martin_sullivan
- [2p]. So What Makes a Transaction:
- http://www.zois.co.uk/acid.html
- [sc]. Scaling:
- http://www.zois.co.uk/scaling.html
- [lx]. Linux:
- http://en.wikipedia.org/wiki/Linux
- [ph]. PHP:
- http://php.net
- [em]. Emacs:
- http://www.gnu.org/software/emacs
- [zt]. ZEND Developer Zone: Extension Writing:
- http://devzone.zend.com/node/view/id/1021
- [sa]. Re-visiting Tuxedo's Simpapp.
- http://www.zois.co.uk/tn/tn-2003-04-14.html
- [gc]. PHP Function
str-getcsv.php. - http://uk.php.net/manual/en/function.str-getcsv.php
- [cc]. Library
libcsv. - http://sourceforge.net/projects/libcsv
~Z~