Blog Archives

Remove comment lines and empty lines from a file

Sometimes you have a config file with a lot of comments and empty lines, which you don’t need, because they hide the relevant information. So use this to simplify it:

cat your.file | grep -v '^#' | grep -v '^$'

 

Frequently used commands of firewall-cmd

Reload firewall settings

firewall-cmd --reload

Interfaces

Bind an interface “eth0” to the default zone.

firewall-cmd --add-interface=eth0 --permanent

Bind an interface “eth0” to a specific zone “public”

firewall-cmd --zone=public --add-interface=eth0 --permanent

Services

Add a service to default zone

firewall-cmd --add-service https --permanent

Add a service to a specific zone “public”

firewall-cmd --zone=public --add-service https --permanent

Ports

Open a port within the default zone

firewall-cmd --add-port 1521/tcp --permanent

Open a port within a specific zone “public”

firewall-cmd --zone=public --add-port 1521/tcp --permanent

Remove a port from a specific zone “public”

firewall-cmd --remove-port 1521/tcp --permanent

Getting Information

List all defined zones

firewall-cmd --get-zones

Get the default zone

firewall-cmd --get-default-zone

List active zones

firewall-cmd --get-active-zones

Get data of a specific zone “public”

firewall-cmd --info-zone=public

 

 

SSH key generation

To authenticate with keys on an SSH session, we need a keypair first. This contains a public and a private key part. The public part must be copied to the SSH server, the private part resides on your user homedir.

ssh-keygen -t rsa -b 4096

This will ask you for the destination of the keyfiles. The file with the extension .pub will be the public key part.

Enter file in which to save the key (/home/<user>/.ssh/id_rsa):
You can leave the default (press Enter-key), of type another file name, i.e. my-ssh-key. Without a path it will be stored into the current working directory.
Now you should secure your private key with an additional keyphrase, which you have to enter on every access to the key. Type it twice and don't forget it.

Enter passphrase (empty for no passphrase):
Enter same passphrase again: 


Your identification has been saved in my-ssh-key
Your public key has been saved in my-ssh-key.pub
The key fingerprint is:
SHA256:Kg4elHNG8TwLIYjTfX7yRz7h0dmVHY7FUx5krwwQjEA user@hostname
The key's randomart image is:
+---[RSA 4096]----+
|.o..+E. o. o++|
|+ ...=.. .. .+oo|
| . oo+ . . *.= |
| o .oo. = o * |
| + o .+S+ + o |
| . + .. = o . |
| o . . . . o |
| . + . |
| . . |
+----[SHA256]-----+

You can move both files into /home/<user>/.ssh/. If the folder doesn’t exist, create it:

mkdir -p ~/.ssh
chmod 700 ~/.ssh

mv my-ssh-key ~/.ssh/.
chmod 600 ~/.ssh/my-ssh-key
chmod 644 ~/.ssh/my-ssh-key.pub

The file permission must be set correctly. Now its time to copy the public key part to the SSH server. You need a working user account there, which can bee reached with a password login.

ssh-copy-id -i ~/.ssh/my-ssh-key user@ssh-hostname

This will copy the content of the my-ssh-key.pub into ~/.ssh/authorized_keys on the SSH server. If you don’t have access to the account (because the SSH server prevents password-based login), ask your administrator. If your keybased login doesn’t work, try on client side

ssh -vvv user@ssh-hostname

to see, what’s going on. It tries some private key names, but the name my-ssh-key (see above) will not used. So we have to configure this in a special file named “config” within ~/.ssh.

cd ~/.ssh
touch config
chmod 644 config

type some SSH parameters into that file.

host <ssh-hostname>
   Hostname <ssh-hostname>
   Port 22
   IdentityFile ~/.ssh/my-ssh-key
   ForwardX11 yes

Replace <ssh-hostname> with the correct name. The important part is IdentityFile, which points to your SSH private key part. ForwardX11 is optional and allows a display redirection from the server to the client for X-based applications. Save the file and try it again:

ssh user@ssh-hostname

This should now ask for the passphrase of the correct key my-ssh-key.

Enter passphrase for key '/home/<user>/.ssh/my-ssh-key':

 

 

 

 

Install WebObjects/Wonder development environment on Linux

I have tried to install a running development environment for WebObjects applications on Linux (OpenSuse 15.4 and Xubuntu 18.04). There are some little problems to solve.

  1. You need an JDK 1.8 on your computer (in parallel with the current JDK11)
  2. You need an installation of ANT
  3. You need an Eclipse IDE.
  4. You need the WebObjects frameworks.
  5. You need WOLips as plugin to Eclipse.
  6. You need the WOnder source for modern applications.

Let’s start with the installations.

zypper install java-1_8_0-openjdk java-1_8_0-openjdk-devel java-1_8_0-openjdk-javadoc java-1_8_0-openjdk-src java-1_8_0-openjdk-headless java-1_8_0-openjdk-demo
zypper install ant

apt install openjdk-8-jdk openjdk-8-demo openjdk-8-doc  openjdk-8-headless openjdk-8-source
apt install ant

Now you have to download the Eclipse IDE (for Java Developers or for Enterprise Java and Web Developers).

https://www.eclipse.org/downloads/packages/

You can install it on /opt (extract the tar.gz there).

I have installed the WebObjects stuff into its own folder called “WODevelopment” within your home folder. There is also the preferred workspace folder for Eclipse.

cd ~
mkdir -p WODevelopment/workspace

Start the IDE with this workspace folder to see any problems. Maybe you have to install some more things you need, like Subclipse.

Now you install the WOLips plugin into Eclipse. Go to Help->Install New Software->Add and create a new location.

WOLips410
https://jenkins.wocommunity.org/job/WOLips410/lastSuccessfulBuild/artifact/temp/dist/

Select all options (the WOLips Goodies are not installable on Linux) and install them. After an IDE restart, you can open a new perspective “WOLips”.

The plugin needs a lot of WebObjects frameworks, which you have to install now. Actually there is an install tool, called WOInstaller, but this doesn’t work for me, it always stops with an exception message. I have tried two versions and end up with a manual installation.

curl -O https://jenkins.wocommunity.org/job/WOInstaller/lastSuccessfulBuild/artifact/Utilities/WOInstall/WOInstaller.jar
curl -O https://wocommunity.org/documents/tools/WOInstaller.jar
java -jar WOInstaller.jar 5.4.3 ~/WODevelopment/Libraries/WOnder

If you get also an exception, try the following.

  1. Go to the GITHUB project of the installer and look for the WebObjectsInstaller.java
    https://github.com/wocommunity/wonder/blob/master/Utilities/WOInstall/Sources/er/woinstaller/WebObjectsInstaller.java
  2. Copy the download link for the latest WebObjects version 5.4.3, which you will find as static value on top of the file
  3. Download the .dmg file (Mac archive)
  4. use 7-zip (7z) to extract the file

You will find a folder “WebObjects Update/Packages”, which contains four further archives (.pkg). These archive files you can also decompress with 7-zip. It generates a “Payload~” archive, which you decompress again.

7z x WebObjectsDevelopment.pkg
7z x Payload~
rm Payload~
7z x WebObjectsDocumentation.pkg
7z x Payload~
rm Payload~
7z x WebObjectsExamples.pkg
7z x Payload~
rm Payload~
7z x WebObjectsRuntime.pkg
7z x Payload~
rm Payload~

Now you have three folders (Developer, Library, System), which you copy into “~/WODevelopment/Libraries/WOnder”.

mkdir -p ~/WODevelopment/Libraries/WOnder
mv Developer ~/WODevelopment/Libraries/WOnder/.
mv Library ~/WODevelopment/Libraries/WOnder/.
mv System ~/WODevelopment/Libraries/WOnder/.

Check the owner of the files and use “chown” if necessary.

Now its time for the global wolips properties file, which contains some settings for the Eclipse plugin and the ANT build pipeline. The file must be on “~/Library/Application Support/WOLips/wolips.properties”.

mkdir -p "~/Library/Application Support/WOLips"
touch wolips.properties

Edit the newly generated file and copy the following properties into it. Change the path prefixes, as necessary, i.e. /home/me/WODevelopment. Don’t use Shortcuts like “~”. Be careful, every path must exist within the filesystem, generate them, if necessary.

wo.system.frameworks=/home/me/WODevelopment/Libraries/WOnder/System/Library/Frameworks
wo.bootstrapjar=/home/me/WODevelopment/Libraries/WOnder/System/Library/WebObjects/JavaApplications/wotaskd.woa/WOBootstrap.jar
wo.extensions=/home/me/WODevelopment/Libraries/WOnder/Library/WebObjects/Extensions
wolips.properties=wolips.properties
wo.system.root=/home/me/WODevelopment/Libraries/WOnder/System
wo.user.frameworks=/home/me/Library/Frameworks
wo.external.root=/home/me/WODevelopment/Libraries/WOnder/External
wo.local.root=/home/me/WODevelopment/Libraries/WOnder
wo.apps.root=/home/me/WODevelopment/Libraries/WOnder/Library/WebObjects/Applications
wo.api.root=/home/me/WODevelopment/Libraries/WOnder/Developer/Documentation/DocSets/com.apple.ADC_Reference_Library.WebObjectsReference.docset/Contents/Resources/Documents/documentation/MacOSXServer/Reference/WO54_Reference
wo.local.frameworks=/home/me/WODevelopment/Libraries/WOnder/Library/Frameworks
wo.network.root=/home/me/WODevelopment/Libraries/WOnder/Network
wo.network.frameworks=/home/me/WODevelopment/Libraries/WOnder/Network/Library/Frameworks
wo.user.root=/home/me
wo.server.root=/home/me/WODevelopment/Libraries/WOnder/Server

The last step is the download and the build of the WOnder source code, the current community extensions to WebObjects.

Go to the GITHUB repository of “WOnder” and download the latest release as ZIP archive.

https://github.com/wocommunity/wonder/releases/latest

Extract the downloaded archive file into the “~/WODevelopment/WonderSource”. Use the tar.gz instead of zip, there is a problem with long filenames.

tar xvzf ~/Downloads/wonder-<version>.tar.gz
mv ~/Downloads/wonder-wonder-<version> ~/WODevelopment/WonderSource

Copy the “wolips.properties” file as “build.properties” into “~/WODevelopment/WonderSource”. It is the config for the following ANT build process.

cd ~/WODevelopment/WonderSource
cp ~/Library/Application Support/WOLips/wolips.properties build.properties

Start the build with JDK1.8 (!). There can be some warnings, but it should end with “BUILD SUCCESSFUL”.

JAVA_HOME=<path to JDK1.8> ant all

Now you can start WebObjects development of your own project within Eclipse. The projects within Eclipse should use Java 1.8 within its Build Path, add this JRE as installed VM.

If you start the first project, you will get an error, that the application cannot be opened within the default browser. Linux is not a supported development platform. So you have to add a special method within your Application.java file:

@Override
public boolean _isSupportedDevelopmentPlatform() {
   return super._isSupportedDevelopmentPlatform() || "Linux".equals(System.getProperty("os.name"));
}

After that, WOLips will call /usr/bin/open to execute the dynamic application URI within the default browser. But this will not work within Linux, but you can define a symbolic link (as root) to your preferred browser:

cd /usr/bin
ln -s /usr/bin/firefox open

Now the browser should automatically display your application within Firefox.

Doctrine ORM with Oracle’s OFFSET and LIMIT

To handle LIMIT and OFFSET within Oracle DBs you have to use some magic. There is a Blog post, which describes the general procedure. Doctrine uses this ROWNUM stuff too, it is implemented within OraclePlatform.php. Doctrine needs two integer values for LIMIT and OFFSET. But you can also set both to NULL, the methods of the Query class (setMaxResults() and setFirstResult()) accept NULL values too.

If you think, it is a good idea to send  PHP_INT_MAX as default for LIMIT, it would be a fail. Within the OraclePlatform.php Doctrine must add the given OFFSET (maybe 20) to the LIMIT (maybe PHP_INT_MAX), so you will run into a datatype overflow. This will result into PHP_INT_MIN, a very large negative number. The resulting SQL would try to filter your ResultSet from OFFSET (20) to a large negative number, it would be empty always (except you will set the OFFSET to 0).

Also, if you set a large LIMIT to force getting all records from a query, you force Doctrine to wrap your query with some of the ROWNUM stuff, which results in a more complex query and an increased query time. Let LIMIT = null, if you need all records. Only set OFFSET > 0 (or != null), if you need the next page of the results.

 

XPath cheats

Get the child element nodes of a book node (without comment nodes and text nodes)

//book/*

Get the child elements, text and comment nodes of a book node

//book/*|text()|comment()

The following XPath queries return the same nodes:

$xpath-&gt;query("./../bookstore", $contextNode)
$xpath-&gt;query("../bookstore", $contextNode);

PDF to PNG

You can convert a multipage PDF into multiple PNG files with a simple Bash statement:

convert -density 300 your.pdf -quality 100 -scale 825x1125 your-%d.png

Now you can change the images with a simple GFX application like Kolourpaint. You can also create new images with the same size and reorder your images by the number within the filename.

After you have finished your work, recombine the PNGs into a PDF:

convert your-*.png your-new.pdf

Java Class Versions

  • Java 1.1 uses major version 45
  • Java 1.2 uses major version 46
  • Java 1.3 uses major version 47
  • Java 1.4 uses major version 48
  • Java 5 uses major version 49
  • Java 6 uses major version 50
  • Java 7 uses major version 51
  • Java 8 uses major version 52
  • Java 9 uses major version 53
  • Java 10 uses major version 54
  • Java 11 uses major version 55
  • Java 12 uses major version 56
  • Java 13 uses major version 57
  • Java 14 uses major version 58

Step 7: CORS

Browsers implement a mechanism called Cross-Origin Resource Sharing (CORS). If your application (client) runs on a domain A, it is only possible to send requests on a domain B with a small set of HTTP verbs.

The domains (origins) differ in domain-name, protocol, and port, so i.e. it will not be possible to send DELETE requests from https://my-domain:443 (where the GUI resides) to https://my-domain:8000 (where we will find the web-service). Some simple non-destructive requests like GETs are possible.

Because of our own Content-Type header, our framework will run out of the CORS safelist and our client will not play with our webservice also for simple GET requests.

So we have to implement the mechanism of preflight requests. The browser will send an OPTIONS request just before the real request to the webservice. The webservice must dis-/allow the requested action and send these information as response to the OPTIONS preflight request. Then the browser will send the real request.

CORS on ERRest

ERRest implements the CORS mechanism within the ERXRouteController.optionsAction(). It is a public method, so every sub-class can overwrite the default implementation.

Some global properties control the access:

ERXRest.accessControlAllowOrigin
This can be set to “*” (for all) or a list of specific origins. The content of the property will be set as value of the Access-Control-Allow-Origin header within the response.

ERXRest.accessControlAllowRequestMethods
The default value of this property is “OPTIONS,GET,HEAD,POST,PUT,DELETE,TRACE,CONNECT”. If you set the property to “” or null, the current requested method will be allowed! The content of the property will be set as value of the Access-Control-Allow-Methods header within the response.

ERXRest.accessControlAllowRequestHeaders
The property contains a comma-separated list of the allowed request headers. If you set the property to “” or null, the requested headers will be allowed! The content of the property will be set as value of the Access-Control-Allow-Headers header within the response.

ERXRest.accessControlMaxAge
The property contains a value in seconds for how long the response to the preflight request can be cached for without sending another preflight request. The default value is 1728000 (20 days). The content of the property (must be greater or equals than 0) will be set as value of the Access-Control-Max-Age header within the response

The values will be retrieved by four methods, which are protected and could be overwritten by the sub-classes of ERXRouteController.

Step 6: performActionNamed

WebObjects’ request-response-loop calls now a method on the instantiated resource class: performActionNamed(). This method gets the already found method name as parameter, WebObjects called this “actionName”. The default implementation would now look for a method called actionNameAction() and would invoke this method with no parameters. The result of the method would be an WOActionResults instance, the response object. This would be sent to the client.

The “actionName” (method name) is not useful for our framework. We could have multiple methods within the resource class with the same name, but different parameter lists or annotations. The simple name without more information cannot be used. But we have already pre-processed the annotated methods during the registration of the resource class. Therefore we should ignore the “actionName” parameter of performActionNamed() and should better use the Method object stored within the ResourceDescription.

The handler class has been stored this description class into the userInfo dictionary of the WORequest object. We can now access this object (the handler has provided the request object over the constructor of the resource class) and extract the method. So the workflow here is:

  1. check the access of the user (is user authenticated, has the user permissions to execute the operation?)
  2. get the parameters for the method, fill the parameters with values according their annotations
  3. translate the request content into an XML and initialize the associated parameters
  4. invoke the method
  5. translate the result of the method into an XML
  6. store the result into the WOResponse
  7. go back to request-response-loop

Check Access

The default implementation of the checkAccess() method within our BaseRestResource class is empty, so we allow every access. But this is not useful for all use cases. The simplest implementation could be the usage of a Basic Authentication mechanism. It is supported by HTTP through the header key “authorization”. It consists of the key word “Basic” followed by a space character and then the Base64 encoded “username:password”:

authorization: Basic YXJvdGhlOm1hcHBl

There are two possible HTTP statuses associated with the authorization. Forbidden (403) should be sent if the authorized user has no permissions on this resource. Unauthorized (401) should be sent, if there are no authorization data available or the request contains the wrong authorization protocol. Both statuses can be reported to the base class by the two exceptions: ForbiddenException and UnauthorizedException. The base class will send the associated error response to the client.

In an own implementation of the checkAccess() method, you can query the role of the user and check the permissions of that role within the application. The example implementation BasicAuthorizationResource checks the username and password for a default user. It stores also the user, so we can later build a filter for the queried business data from the database. Not every user should see all data, so it would be necessary to hide some information per user or role.

Collect Parameter Values

We implement a method getParametersForMethod(), which extracts the annotations of the given method. The values for @HeaderParam and @CookieParam can be get from the request object.

protected String getCookieValueForKey(String key, String def) {
   String val = request().cookieValueForKey(key);
   return val == null ? def : val;
}

protected String getHeaderValueForKey(String key, String def) {
   return request().headerForKey(key, def);
}

The values from the query string (the part of the URI behind the “?”) must be parsed, the name of the parameter is defined in @QueryParam:

protected String getQueryValueForKey(String key String def) {
   String query = request().queryString();
   String[] parts = query.split("&amp;");
   for (String p : parts) {
      if (!p.startsWith(key + "=")) {
         continue;
      }
      try {
         return URLDecoder.decode(p.substring(key.length() + 1), "UTF-8");
      } catch (Throwable t) {
         return def;
      }
   }
   return def;
}

For @PathParam we have already pre-processed some things. We can use the regular expression of the ResourceDescription and we have already built a Map with the parameter names and their matching group numbers within the regular expression.

protected String getPathParameterValueForKey(String key, String def) {
   ResourceDescription dscr = getResourceDescription();
   Integer idx = dscr.getPathParameters().get(key);
   if (idx == null) {
      return def;
   }
   try {
      Pattern pattern = Pattern.compile(dscr.getRegex());
      Matcher matcher = pattern.matcher(request().requestHandlerPath());
      if (!matcher.find()) {
         return def;
      }
      return matcher.group(idx);
   } catch (Throwable t) {
      return def;
   }
}

The methods above use also a possible defined @DefaultValue, which we extract with the following code (p is the Parameter instance from the method):

DefaultValue dv = p.getAnnotation(DefaultValue.class);
String def = dv == null ? null : dv.value();

All methods return String values for the parameters. But we have to instantiate the classes given by each parameter type. Allowed are some primitive types (like int, long, double, boolean), classes with a constructor which needs a single String parameter, classes with static method valueOf(String) or List<String>. To cast the String object into the necessary parameter type, we use the method buildObjectFromString():

protected Object buildObjectFromString(String str, Class<?&gt; type) {
   if (str == null) {
      return null;
   }
   if (type.isPrimitive()) {
      if (type == Boolean.TYPE) {
         return Boolean.valueOf(str);
      }
      if (type == Integer.TYPE) {
         return Integer.valueOf(str);
      }
      if (type == Long.TYPE) {
         return Long.valueOf(str);
      }
      if (type == Float.TYPE) {
         return Float.valueOf(str);
      }
      if (type == Double.TYPE) {
         return Double.valueOf(str);
      }
      if (type == Character.TYPE) {
         return Character.valueOf(str.charAt(0));
      }
      if (type == Byte.TYPE) {
         return Byte.valueOf(str);
      }
   }
   try {
      Constructor<?&gt; cons = type.getConstructor(String.class);
      if (cons != null) {
         return cons.newInstance(str);
      }
   } catch (Throwable t) {
         // do nothing
   }
   try {
	Method m = type.getMethod("valueOf", String.class);
        if (m != null &amp;&amp; Modifier.isStatic(m.getModifiers())) {
           return m.invoke(null, str);
        }
   } catch (Throwable t) {
      // do nothing
   }
   List<String&gt; res = new ArrayList<String&gt;();
   if (type.isAssignableFrom(res.getClass())) {
      res.add(str);
      return res;
   }
   log.debug("cannot assign String \"" + str + "\" to the given type \"" + type.getName() + "\"");
   return null;
}

The objects will be added to an array, which we can use on invoking the action method. We await a further optional parameter, which we use to transport the request body content. If there is no content, all further parameters without annotations will be null.

Translate The XML-Content

To prevent a lot of self-written code, we will use JAXB as a framework to translate a POJO into an XML representation and vice versa. So we use so called message classes to transport the data from the request into our Resource class.

The method getParametersForMethod() from AbstractRestResource try to assign data to the method parameters. Some parameters will be annotated, the data will come from the request object. The message parameters will be filled by the request content. If there is no content, the parameters becomes null. Every parameter without an annotation should have a type which the JAXBContext can unmarshal (translate from XML to class instance). If it is not possible, the parameter will be null.

It could be possible, that a parameter without annotation has a primitive type, so an assignment of null will be wrong. This condition will result in an exception, which is reported to the client as internal error of the service.

The result of the called resource method should be “void” (no response content) or a message type, which the JAXBContext can marshal into an XML string. So we can fill the POJO within the method and return it to the caller (Rest handler). If the return type cannot by used by JAXB, it results in an exception, which will be reported as internal error to the client.

At the moment, the implementation doesn’t check the annotated @Produces/@Consumes against the type of the parameters. Because the message classes define the static MEDIA_TYPE, it could be possible to compare the annotation value against the static MEDIA_TYPE string. We would always need such a MEDIA_TYPE string within the message class, which I would avoid. Also the “Content-Type” response header will be set to the @Produces value and not to the possible available static MEDIA_TYPE member field of the method’s return value.

Create the response

As response, the Rest handler class will generate always an WOResponse instance with an HTTP code of 200 (OK). The “Content-Type” header will be set to @Produces value and the content of the request is filled with the return value of the method (XML string built from a POJO).

In the case of an exception, the response has an error code of 500 (internal error) and the response content should contain a description of the error. We can log the stack trace, but the client should always get a human readable description (i.e. from a ResourceBundle), which the client can show on a GUI. We will discuss this within the Blog post about the exception handling.

Back to request-response-loop

The created response instance is the return value of the method performActionNamed() within the RestResource class (which has been called by the request handler class). We should always prevent any exceptions on this level, because the RRL will handle exceptions with its own error responses. These responses cannot be handled by our Rest-based client.

The returned WOResponse object will be sent to the client and the loop will end for this request. Because our service doesn’t use a session (stateless service), a lot of other code within WOActionRequestHandler._handleRequest() will be ignored.