Doctrine ORM with Oracle’s OFFSET and LIMIT

To handle LIMIT and OFFSET within Oracle DBs you have to use some magic. There is a Blog post, which describes the general procedure. Doctrine uses this ROWNUM stuff too, it is implemented within OraclePlatform.php. Doctrine needs two integer values for LIMIT and OFFSET. But you can also set both to NULL, the methods of the Query class (setMaxResults() and setFirstResult()) accept NULL values too.

If you think, it is a good idea to send  PHP_INT_MAX as default for LIMIT, it would be a fail. Within the OraclePlatform.php Doctrine must add the given OFFSET (maybe 20) to the LIMIT (maybe PHP_INT_MAX), so you will run into a datatype overflow. This will result into PHP_INT_MIN, a very large negative number. The resulting SQL would try to filter your ResultSet from OFFSET (20) to a large negative number, it would be empty always (except you will set the OFFSET to 0).

Also, if you set a large LIMIT to force getting all records from a query, you force Doctrine to wrap your query with some of the ROWNUM stuff, which results in a more complex query and an increased query time. Let LIMIT = null, if you need all records. Only set OFFSET > 0 (or != null), if you need the next page of the results.

 

Access private Gitlab repo with Composer

To access a project (like a library) on a private repository within your own Gitlab server, it is necessary to configure Composer right. First add the repository to the composer.json file of the current project (where you need the library code):

{
"repositories": [{
"type": "gitlab",
"url": "https://gitlab.yourserver.com/user/project.git" }]

The type “gitlab” activates some more configuration properties for Composer. So you have to define your private Gitlab server as valid server within Composer:

composer config --global gitlab-domains "gitlab.yourserver.com"

The –global flag stores the property within ~/.config/composer/config.json (OpenSuse 15.2). It will be activate for every Composer project on the computer.

If your repository is only visible after the login with a user/password, you must generate a unique token within the GitLab. Log into the GitLab UI with your username/password and choose “Preferences”->”Access Tokens”. There you can enter an application name like “PHP composer” and select “read_api” permissions. You get a new token, which you now can use within Composer config.

It is necessary to define the username and the access token for your Gitlab account, to provide access to the repository:

composer config gitlab-token.username "your GitLab username"
composer config gitlab-token.gitlab.yourserver.com "generated token"

Here you could also add –global or leave it and define the token only for the current project. The lines above should generate a file auth.json within your current project:


{
"gitlab-token": {
"username": "your GitLab username",
"gitlab.yourserver.com": "generated token"
}
}

Now you need a prepared repository on “https://gitlab.yourserver.com/user/project.git” (the library project). On the top level of the master branch there must be a composer.json file, which should not contain a version number, but a valid “name” property like (and “require” and “autoload” sections):


{
"name": "company/private_library"
}

The possible versions of your library will be extracted from the tags or branches of the project. Some Information about versions you can find on the composer website. The simplest way is to generate a tag of your latest commit and name it like “v1.0.0”. Now it is possible to define the library repository as a requirement within your current project:


{
"require": {
"company/private_library": "1.0.*"
}
}

You can use the version handling of Composer, if your tags will have valid version numbers. It is also possible to use stability constraints like “stable” or “dev”, use it within your tag name like “v1.0.0-dev”


{
"require": {
"company/private_library": "1.0.*@dev"
}
}

A following composer update should clone the repository into your vendor directory and the autoloader can provide the library project classes.

XPath cheats

Get the child element nodes of a book node (without comment nodes and text nodes)

//book/*

Get the child elements, text and comment nodes of a book node

//book/*|text()|comment()

The following XPath queries return the same nodes:

$xpath->query("./../bookstore", $contextNode)
$xpath->query("../bookstore", $contextNode);

PDF to PNG

You can convert a multipage PDF into multiple PNG files with a simple Bash statement:

convert -density 300 your.pdf -quality 100 -scale 825x1125 your-%d.png

Now you can change the images with a simple GFX application like Kolourpaint. You can also create new images with the same size and reorder your images by the number within the filename.

After you have finished your work, recombine the PNGs into a PDF:

convert your-*.png your-new.pdf

Java Class Versions

  • Java 1.1 uses major version 45
  • Java 1.2 uses major version 46
  • Java 1.3 uses major version 47
  • Java 1.4 uses major version 48
  • Java 5 uses major version 49
  • Java 6 uses major version 50
  • Java 7 uses major version 51
  • Java 8 uses major version 52
  • Java 9 uses major version 53
  • Java 10 uses major version 54
  • Java 11 uses major version 55
  • Java 12 uses major version 56
  • Java 13 uses major version 57
  • Java 14 uses major version 58

Step 7: CORS

Browsers implement a mechanism called Cross-Origin Resource Sharing (CORS). If your application (client) runs on a domain A, it is only possible to send requests on a domain B with a small set of HTTP verbs.

The domains (origins) differ in domain-name, protocol, and port, so i.e. it will not be possible to send DELETE requests from https://my-domain:443 (where the GUI resides) to https://my-domain:8000 (where we will find the web-service). Some simple non-destructive requests like GETs are possible.

Because of our own Content-Type header, our framework will run out of the CORS safelist and our client will not play with our webservice also for simple GET requests.

So we have to implement the mechanism of preflight requests. The browser will send an OPTIONS request just before the real request to the webservice. The webservice must dis-/allow the requested action and send these information as response to the OPTIONS preflight request. Then the browser will send the real request.

CORS on ERRest

ERRest implements the CORS mechanism within the ERXRouteController.optionsAction(). It is a public method, so every sub-class can overwrite the default implementation.

Some global properties control the access:

ERXRest.accessControlAllowOrigin
This can be set to “*” (for all) or a list of specific origins. The content of the property will be set as value of the Access-Control-Allow-Origin header within the response.

ERXRest.accessControlAllowRequestMethods
The default value of this property is “OPTIONS,GET,HEAD,POST,PUT,DELETE,TRACE,CONNECT”. If you set the property to “” or null, the current requested method will be allowed! The content of the property will be set as value of the Access-Control-Allow-Methods header within the response.

ERXRest.accessControlAllowRequestHeaders
The property contains a comma-separated list of the allowed request headers. If you set the property to “” or null, the requested headers will be allowed! The content of the property will be set as value of the Access-Control-Allow-Headers header within the response.

ERXRest.accessControlMaxAge
The property contains a value in seconds for how long the response to the preflight request can be cached for without sending another preflight request. The default value is 1728000 (20 days). The content of the property (must be greater or equals than 0) will be set as value of the Access-Control-Max-Age header within the response

The values will be retrieved by four methods, which are protected and could be overwritten by the sub-classes of ERXRouteController.

Step 6: performActionNamed

WebObjects’ request-response-loop calls now a method on the instantiated resource class: performActionNamed(). This method gets the already found method name as parameter, WebObjects called this “actionName”. The default implementation would now look for a method called actionNameAction() and would invoke this method with no parameters. The result of the method would be an WOActionResults instance, the response object. This would be sent to the client.

The “actionName” (method name) is not useful for our framework. We could have multiple methods within the resource class with the same name, but different parameter lists or annotations. The simple name without more information cannot be used. But we have already pre-processed the annotated methods during the registration of the resource class. Therefore we should ignore the “actionName” parameter of performActionNamed() and should better use the Method object stored within the ResourceDescription.

The handler class has been stored this description class into the userInfo dictionary of the WORequest object. We can now access this object (the handler has provided the request object over the constructor of the resource class) and extract the method. So the workflow here is:

  1. check the access of the user (is user authenticated, has the user permissions to execute the operation?)
  2. get the parameters for the method, fill the parameters with values according their annotations
  3. translate the request content into an XML and initialize the associated parameters
  4. invoke the method
  5. translate the result of the method into an XML
  6. store the result into the WOResponse
  7. go back to request-response-loop

Check Access

The default implementation of the checkAccess() method within our BaseRestResource class is empty, so we allow every access. But this is not useful for all use cases. The simplest implementation could be the usage of a Basic Authentication mechanism. It is supported by HTTP through the header key “authorization”. It consists of the key word “Basic” followed by a space character and then the Base64 encoded “username:password”:

authorization: Basic YXJvdGhlOm1hcHBl

There are two possible HTTP statuses associated with the authorization. Forbidden (403) should be sent if the authorized user has no permissions on this resource. Unauthorized (401) should be sent, if there are no authorization data available or the request contains the wrong authorization protocol. Both statuses can be reported to the base class by the two exceptions: ForbiddenException and UnauthorizedException. The base class will send the associated error response to the client.

In an own implementation of the checkAccess() method, you can query the role of the user and check the permissions of that role within the application. The example implementation BasicAuthorizationResource checks the username and password for a default user. It stores also the user, so we can later build a filter for the queried business data from the database. Not every user should see all data, so it would be necessary to hide some information per user or role.

Collect Parameter Values

We implement a method getParametersForMethod(), which extracts the annotations of the given method. The values for @HeaderParam and @CookieParam can be get from the request object.

protected String getCookieValueForKey(String key, String def) {
   String val = request().cookieValueForKey(key);
   return val == null ? def : val;
}

protected String getHeaderValueForKey(String key, String def) {
   return request().headerForKey(key, def);
}

The values from the query string (the part of the URI behind the “?”) must be parsed, the name of the parameter is defined in @QueryParam:

protected String getQueryValueForKey(String key String def) {
   String query = request().queryString();
   String[] parts = query.split("&");
   for (String p : parts) {
      if (!p.startsWith(key + "=")) {
         continue;
      }
      try {
         return URLDecoder.decode(p.substring(key.length() + 1), "UTF-8");
      } catch (Throwable t) {
         return def;
      }
   }
   return def;
}

For @PathParam we have already pre-processed some things. We can use the regular expression of the ResourceDescription and we have already built a Map with the parameter names and their matching group numbers within the regular expression.

protected String getPathParameterValueForKey(String key, String def) {
   ResourceDescription dscr = getResourceDescription();
   Integer idx = dscr.getPathParameters().get(key);
   if (idx == null) {
      return def;
   }
   try {
      Pattern pattern = Pattern.compile(dscr.getRegex());
      Matcher matcher = pattern.matcher(request().requestHandlerPath());
      if (!matcher.find()) {
         return def;
      }
      return matcher.group(idx);
   } catch (Throwable t) {
      return def;
   }
}

The methods above use also a possible defined @DefaultValue, which we extract with the following code (p is the Parameter instance from the method):

DefaultValue dv = p.getAnnotation(DefaultValue.class);
String def = dv == null ? null : dv.value();

All methods return String values for the parameters. But we have to instantiate the classes given by each parameter type. Allowed are some primitive types (like int, long, double, boolean), classes with a constructor which needs a single String parameter, classes with static method valueOf(String) or List<String>. To cast the String object into the necessary parameter type, we use the method buildObjectFromString():

protected Object buildObjectFromString(String str, Class<?&gt; type) {
   if (str == null) {
      return null;
   }
   if (type.isPrimitive()) {
      if (type == Boolean.TYPE) {
         return Boolean.valueOf(str);
      }
      if (type == Integer.TYPE) {
         return Integer.valueOf(str);
      }
      if (type == Long.TYPE) {
         return Long.valueOf(str);
      }
      if (type == Float.TYPE) {
         return Float.valueOf(str);
      }
      if (type == Double.TYPE) {
         return Double.valueOf(str);
      }
      if (type == Character.TYPE) {
         return Character.valueOf(str.charAt(0));
      }
      if (type == Byte.TYPE) {
         return Byte.valueOf(str);
      }
   }
   try {
      Constructor<?&gt; cons = type.getConstructor(String.class);
      if (cons != null) {
         return cons.newInstance(str);
      }
   } catch (Throwable t) {
         // do nothing
   }
   try {
	Method m = type.getMethod("valueOf", String.class);
        if (m != null &amp;&amp; Modifier.isStatic(m.getModifiers())) {
           return m.invoke(null, str);
        }
   } catch (Throwable t) {
      // do nothing
   }
   List<String&gt; res = new ArrayList<String&gt;();
   if (type.isAssignableFrom(res.getClass())) {
      res.add(str);
      return res;
   }
   log.debug("cannot assign String \"" + str + "\" to the given type \"" + type.getName() + "\"");
   return null;
}

The objects will be added to an array, which we can use on invoking the action method. We await a further optional parameter, which we use to transport the request body content. If there is no content, all further parameters without annotations will be null.

Translate The XML-Content

To prevent a lot of self-written code, we will use JAXB as a framework to translate a POJO into an XML representation and vice versa. So we use so called message classes to transport the data from the request into our Resource class.

The method getParametersForMethod() from AbstractRestResource try to assign data to the method parameters. Some parameters will be annotated, the data will come from the request object. The message parameters will be filled by the request content. If there is no content, the parameters becomes null. Every parameter without an annotation should have a type which the JAXBContext can unmarshal (translate from XML to class instance). If it is not possible, the parameter will be null.

It could be possible, that a parameter without annotation has a primitive type, so an assignment of null will be wrong. This condition will result in an exception, which is reported to the client as internal error of the service.

The result of the called resource method should be “void” (no response content) or a message type, which the JAXBContext can marshal into an XML string. So we can fill the POJO within the method and return it to the caller (Rest handler). If the return type cannot by used by JAXB, it results in an exception, which will be reported as internal error to the client.

At the moment, the implementation doesn’t check the annotated @Produces/@Consumes against the type of the parameters. Because the message classes define the static MEDIA_TYPE, it could be possible to compare the annotation value against the static MEDIA_TYPE string. We would always need such a MEDIA_TYPE string within the message class, which I would avoid. Also the “Content-Type” response header will be set to the @Produces value and not to the possible available static MEDIA_TYPE member field of the method’s return value.

Create the response

As response, the Rest handler class will generate always an WOResponse instance with an HTTP code of 200 (OK). The “Content-Type” header will be set to @Produces value and the content of the request is filled with the return value of the method (XML string built from a POJO).

In the case of an exception, the response has an error code of 500 (internal error) and the response content should contain a description of the error. We can log the stack trace, but the client should always get a human readable description (i.e. from a ResourceBundle), which the client can show on a GUI. We will discuss this within the Blog post about the exception handling.

Back to request-response-loop

The created response instance is the return value of the method performActionNamed() within the RestResource class (which has been called by the request handler class). We should always prevent any exceptions on this level, because the RRL will handle exceptions with its own error responses. These responses cannot be handled by our Rest-based client.

The returned WOResponse object will be sent to the client and the loop will end for this request. Because our service doesn’t use a session (stateless service), a lot of other code within WOActionRequestHandler._handleRequest() will be ignored.

Step 5: Context

The next step builds the WOContext with data from the WORequest. The WOAction class (which is the abstract base class of WODirectAction which again is the base class for our resource classes) delegates the task to the WOApplication. Because the REST service is stateless and doesn’t generate webpages, we don’t change this process. The Request-Response-Loop can call the default implementation.

Step 4: getActionInstance

This method will be called by the Request-Response-Loop to instantiate the resource class. The default implementation invokes the constructor of the given action class with the WORequest instance.

ERRest uses the method to initialize the newly created class. In our framework, there is nothing to do at the moment. We can use the default implementation. Maybe we can use other constructors here to set URIBuilder, which we need later for the Ids within the XMLMessages.

@SuppressWarnings("rawtypes")
@Override
public WOAction getActionInstance(Class anActionClass, Class[] parameters, Object[] arguments) {
   BaseRestResource actionResource = (BaseRestResource) super.getActionInstance(anActionClass, parameters, arguments);

		
   // TODO: init more things here


   return actionResource;
}

The parameters of the method have been built by the getRequestActionClassAndNameForPath() – see last post.

Step 3: getRequestActionClassAndNameForPath

This method is very simple, it gets the NSArray<String> with class name and method name of the resource class. And it translates the data into an array of three Objects. The third object will be the Class for the given class name.

@Override
public Object[] getRequestActionClassAndNameForPath(@SuppressWarnings("rawtypes") NSArray requestHandlerPath) {
   String requestActionClassName = (String) requestHandlerPath.objectAtIndex(0);
   String requestActionName = (String) requestHandlerPath.objectAtIndex(1);
   return new Object[] { requestActionClassName, requestActionName, _NSUtilities.classWithName(requestActionClassName) };
}

The method name will be called “actionName”, because the normal behaviour of WODirectActionHandler adds the term “Action” at the end of the method name and looks for such a method within the class. But we don’t longer support that.

The method implementation is exact the same as in ERRest.