Add type sanitation

README.md

@@ -652,10 +652,13 @@ You can tune the middleware behavior using middleware specific configuration par
 - "authorization.columnHandler": Handler to implement column authorization rules ("")
 - "authorization.recordHandler": Handler to implement record authorization filter rules ("")
 - "validation.handler": Handler to implement validation rules for input values ("")
-- "validation.types": List of types for which the default validation must take place ("all")
+- "validation.types": Types to enable type validation for, empty means 'none' ("all")
+- "validation.tables": Tables to enable type validation for, empty means 'none' ("all")
 - "ipAddress.tables": Tables to search for columns to override with IP address ("")
 - "ipAddress.columns": Columns to protect and override with the IP address on create ("")
 - "sanitation.handler": Handler to implement sanitation rules for input values ("")
+- "sanitation.types": Types to enable type sanitation for, empty means 'none' ("all")
+- "sanitation.tables": Tables to enable type sanitation for, empty means 'none' ("all")
 - "multiTenancy.handler": Handler to implement simple multi-tenancy rules ("")
 - "pageLimits.pages": The maximum page number that a list operation allows ("100")
 - "pageLimits.records": The maximum number of records returned by a list operation ("1000")
@@ -887,14 +890,28 @@ the 'sanitation' middleware and define a 'sanitation.handler' function that retu
 
 The above example will strip all HTML tags from strings in the input.
 
-### Validating input
+### Type sanitation
+
+If you enable the 'sanitation' middleware, then you (automtically) also enable type sanitation. When this is enabled you may:
+
+- send leading and trailing whitespace (it will be ignored).
+- send a float to an integer field (it will be rounded).
+- send a base64url encoded (it will be converted to regular base64 encoding).
+- send a time/date/timestamp in any strtotime accepted format (it will be converted).
 
-By default all input is accepted unless the validation middleware is specified. The default types validations are then applied.
+You may use the config settings "`sanitation.types`" and "`sanitation.tables`"' to define for which types and
+in which tables you want to apply type sanitation (defaults to 'all'). Example:
 
-#### Validation handler
+    'sanitation.types' => 'date,timestamp',
+    'sanitation.tables' => 'posts,comments',
 
-If you want to validate the input in a custom way, you may add the 'validation' middleware and define a 'validation.handler' 
-function that returns a boolean indicating whether or not the value is valid.
+Here we enable the type sanitation for date and timestamp fields in the posts and comments tables.
+
+### Validating input
+
+By default all input is accepted and sent to the database. If you want to validate the input in a custom way, 
+you may add the 'validation' middleware and define a 'validation.handler' function that returns a boolean 
+indicating whether or not the value is valid.
 
     'validation.handler' => function ($operation, $tableName, $column, $value, $context) {
         return ($column['name'] == 'post_id' && !is_numeric($value)) ? 'must be numeric' : true;
@@ -920,9 +937,10 @@ Then the server will return a '422' HTTP status code and nice error message:
 
 You can parse this output to make form fields show up with a red border and their appropriate error message.
 
-#### Validation types
+### Type validations
 
-The default types validations return the following error messages:
+If you enable the 'validation' middleware, then you (automtically) also enable type validation. 
+This includes the following error messages:
 
 | error message       | reason                      | applies to types                            |
 | ------------------- | --------------------------- | ------------------------------------------- |
@@ -940,18 +958,13 @@ The default types validations return the following error messages:
 | invalid timestamp   | use yyyy-mm-dd hh:mm:ss     | timestamp                                   |
 | invalid base64      | illegal characters          | varbinary, blob                             |
 
-If you want the types validation to apply to all the types, you must activate the "`validation`" middleware.
-By default, all types are enabled. Which is equivalent to the two configuration possibilities:
-
-    'validation.types' => 'all',
-    
-or
-
-    'validation.types'=> 'integer,bigint,varchar,decimal,float,double,boolean,date,time,timestamp,clob,blob,varbinary,geometry',
+You may use the config settings "`validation.types`" and "`validation.tables`"' to define for which types and
+in which tables you want to apply type validation (defaults to 'all'). Example:
 
-In case you want to use a validation handler but don't want any types validation, use:
+    'validation.types' => 'date,timestamp',
+    'validation.tables' => 'posts,comments',
 
-    'validation.types' => '',
+Here we enable the type validation for date and timestamp fields in the posts and comments tables.
 
 NB: Types that are enabled will be checked for null values when the column is non-nullable.
 
@@ -1058,41 +1071,30 @@ in case that you use a non-default "cacheType" the hostname (optionally with por
 
 ## Types
 
-These are the supported types with their default length/precision/scale:
-
-character types
-- varchar(255)
-- clob
-
-boolean types:
-- boolean
-
-integer types:
-- integer
-- bigint
-
-floating point types:
-- float
-- double
-
-decimal types:
-- decimal(19,4)
-
-date/time types:
-- date
-- time
-- timestamp
-
-binary types:
-- varbinary(255)
-- blob
-
-other types:
-- geometry /* non-jdbc type, extension with limited support */
+These are the supported types with their length, category, JSON type and format:
+
+| type       | length | category  | JSON type | format              |
+| ---------- | ------ | --------- | --------- | ------------------- |
+| varchar    | 255    | character | string    |                     |
+| clob       |        | character | string    |                     |
+| boolean    |        | boolean   | boolean   |                     |
+| integer    |        | integer   | number    |                     |
+| bigint     |        | integer   | number    |                     |
+| float      |        | float     | number    |                     |
+| double     |        | float     | number    |                     |
+| decimal    | 19,4   | decimal   | string    |                     |
+| date       |        | date/time | string    | yyyy-mm-dd          | 
+| time       |        | date/time | srting    | hh:mm:ss            |
+| timestamp  |        | date/time | string    | yyyy-mm-dd hh:mm:ss |
+| varbinary  | 255    | binary    | string    | base64 encoded      |
+| blob       |        | binary    | string    | base64 encoded      |
+| geometry   |        | other     | string    | well-known text     |
+
+Note that geometry is a non-jdbc type and thus has limited support.
 
 ## Data types in JavaScript
 
-Javascript and Javascript object notation are not very well suited for reading database records. Decimal, date/time, binary and geometry types are represented as strings in JSON (binary is base64 encoded, geometries are in WKT format). Below are two more serious issues described.
+Javascript and Javascript object notation (JSON) are not very well suited for reading database records. Decimal, date/time, binary and geometry types must be represented as strings in JSON (binary is base64 encoded, geometries are in WKT format). Below are two more serious issues described.
 
 ### 64 bit integers
 
@@ -1147,7 +1149,7 @@ I am testing mainly on Ubuntu and I have the following test setups:
   - (Docker) Debian 9 with PHP 7.0, MariaDB 10.1, PostgreSQL 9.6 (PostGIS 2.3) and SQLite 3.16
   - (Docker) Ubuntu 18.04 with PHP 7.2, MySQL 5.7, PostgreSQL 10.4 (PostGIS 2.4) and SQLite 3.22
   - (Docker) Debian 10 with PHP 7.3, MariaDB 10.3, PostgreSQL 11.4 (PostGIS 2.5) and SQLite 3.27
-  - (Docker) Ubuntu 20.04 with PHP 7.3, MySQL 8.0, PostgreSQL 12.2 (PostGIS 3.0) and SQLite 3.31
+  - (Docker) Ubuntu 20.04 with PHP 7.4, MySQL 8.0, PostgreSQL 12.2 (PostGIS 3.0) and SQLite 3.31
   - (Docker) CentOS 8 with PHP 7.4, MariaDB 10.4, PostgreSQL 12.2 (PostGIS 3.0) and SQLite 3.26
 
 This covers not all environments (yet), so please notify me of failing tests and report your environment. 
@@ -1262,7 +1264,7 @@ To run the docker tests run "build_all.sh" and "run_all.sh" from the docker dire
     sqlsrv: skipped, driver not loaded
     sqlite: 105 tests ran in 1063 ms, 12 skipped, 0 failed
     ================================================
-    Ubuntu 20.04 (PHP 7.3)
+    Ubuntu 20.04 (PHP 7.4)
     ================================================
     [1/4] Starting MySQL 8.0 ........ done
     [2/4] Starting PostgreSQL 12.2 .. done

api.php

@@ -8155,6 +8155,7 @@ namespace Tqdev\PhpCrudApi\Middleware {
     use Psr\Http\Message\ServerRequestInterface;
     use Psr\Http\Server\RequestHandlerInterface;
     use Tqdev\PhpCrudApi\Column\Reflection\ReflectedTable;
+    use Tqdev\PhpCrudApi\Column\Reflection\ReflectedColumn;
     use Tqdev\PhpCrudApi\Column\ReflectionService;
     use Tqdev\PhpCrudApi\Controller\Responder;
     use Tqdev\PhpCrudApi\Middleware\Base\Middleware;
@@ -8179,11 +8180,97 @@ namespace Tqdev\PhpCrudApi\Middleware {
                 if ($table->hasColumn($columnName)) {
                     $column = $table->getColumn($columnName);
                     $value = call_user_func($handler, $operation, $tableName, $column->serialize(), $value);
+                    $value = $this->sanitizeType($table, $column, $value);
                 }
             }
             return (object) $context;
         }
 
+        private function sanitizeType(ReflectedTable $table, ReflectedColumn $column, $value)
+        {
+            $tables = $this->getArrayProperty('tables', 'all');
+            $types = $this->getArrayProperty('types', 'all');
+            if (
+                (in_array('all', $tables) || in_array($table->getName(), $tables)) &&
+                (in_array('all', $types) || in_array($column->getType(), $types))
+            ) {
+                if (is_null($value)) {
+                    return $value;
+                }
+                if (is_string($value)) {
+                    $newValue = null;
+                    switch ($column->getType()) {
+                        case 'integer':
+                        case 'bigint':
+                            $newValue = filter_var(trim($value), FILTER_VALIDATE_INT, FILTER_NULL_ON_FAILURE);
+                            break;
+                        case 'decimal':
+                            $newValue = filter_var(trim($value), FILTER_VALIDATE_FLOAT, FILTER_NULL_ON_FAILURE);
+                            if (is_float($newValue)) {
+                                $newValue = number_format($newValue, $column->getScale(), '.', '');
+                            }
+                            break;
+                        case 'float':
+                        case 'double':
+                            $newValue = filter_var(trim($value), FILTER_VALIDATE_FLOAT, FILTER_NULL_ON_FAILURE);
+                            break;
+                        case 'boolean':
+                            $newValue = filter_var(trim($value), FILTER_VALIDATE_BOOLEAN, FILTER_NULL_ON_FAILURE);
+                            break;
+                        case 'date':
+                            $time = strtotime(trim($value));
+                            if ($time !== false) {
+                                $newValue = date('Y-m-d', $time);
+                            }
+                            break;
+                        case 'time':
+                            $time = strtotime(trim($value));
+                            if ($time !== false) {
+                                $newValue = date('H:i:s', $time);
+                            }
+                            break;
+                        case 'timestamp':
+                            $time = strtotime(trim($value));
+                            if ($time !== false) {
+                                $newValue = date('Y-m-d H:i:s', $time);
+                            }
+                            break;
+                        case 'blob':
+                        case 'varbinary':
+                            // allow base64url format
+                            $newValue = strtr(trim($value), '-_', '+/');
+                            break;
+                        case 'clob':
+                        case 'varchar':
+                            $newValue = $value;
+                            break;
+                        case 'geometry':
+                            $newValue = trim($value);
+                            break;
+                    }
+                    if (!is_null($newValue)) {
+                        $value = $newValue;
+                    }
+                } else {
+                    switch ($column->getType()) {
+                        case 'integer':
+                        case 'bigint':
+                            if (is_float($value)) {
+                                $value = (int) round($value);
+                            }
+                            break;
+                        case 'decimal':
+                            if (is_float($value) || is_int($value)) {
+                                $value = number_format((float) $value, $column->getScale(), '.', '');
+                            }
+                            break;
+                    }
+                }
+                // post process
+            }
+            return $value;
+        }
+
         public function process(ServerRequestInterface $request, RequestHandlerInterface $next): ResponseInterface
         {
             $operation = RequestUtils::getOperation($request);
@@ -8247,7 +8334,7 @@ namespace Tqdev\PhpCrudApi\Middleware {
     				$column = $table->getColumn($columnName);
     				$valid = call_user_func($handler, $operation, $tableName, $column->serialize(), $value, $context);
     				if ($valid === true || $valid === '') {
-    					$valid = $this->validateType($column, $value);
+    					$valid = $this->validateType($table, $column, $value);
     				}
     				if ($valid !== true && $valid !== '') {
     					$details[$columnName] = $valid;
@@ -8260,10 +8347,14 @@ namespace Tqdev\PhpCrudApi\Middleware {
     		return null;
     	}
 
-    	private function validateType(ReflectedColumn $column, $value)
+    	private function validateType(ReflectedTable $table, ReflectedColumn $column, $value)
     	{
+    		$tables = $this->getArrayProperty('tables', 'all');
     		$types = $this->getArrayProperty('types', 'all');
-    		if (in_array('all', $types) || in_array($column->getType(), $types)) {
+    		if (
+    			(in_array('all', $tables) || in_array($table->getName(), $tables)) &&
+    			(in_array('all', $types) || in_array($column->getType(), $types))
+    		) {
     			if (is_null($value)) {
     				return ($column->getNullable() ? true : "cannot be null");
     			}
@@ -8290,15 +8381,24 @@ namespace Tqdev\PhpCrudApi\Middleware {
     							return 'invalid integer';
     						}
     						break;
-    					case 'varchar':
-    						if (mb_strlen($value, 'UTF-8') > $column->getLength()) {
-    							return 'string too long';
-    						}
-    						break;
     					case 'decimal':
-    						if (!is_numeric($value)) {
+    						if (strpos($value, '.') !== false) {
+    							list($whole, $decimals) = explode('.', $value, 2);
+    						} else {
+    							list($whole, $decimals) = array($value, '');
+    						}
+    						if (strlen($whole) > 0 && !ctype_digit($whole)) {
+    							return 'invalid decimal';
+    						}
+    						if (strlen($decimals) > 0 && !ctype_digit($decimals)) {
     							return 'invalid decimal';
     						}
+    						if (strlen($whole) > $column->getPrecision() - $column->getScale()) {
+    							return 'decimal too large';
+    						}
+    						if (strlen($decimals) > $column->getScale()) {
+    							return 'decimal too precise';
+    						}
     						break;
     					case 'float':
     					case 'double':
@@ -8310,9 +8410,7 @@ namespace Tqdev\PhpCrudApi\Middleware {
     						}
     						break;
     					case 'boolean':
-    						if (
-    							filter_var($value, FILTER_VALIDATE_BOOLEAN, FILTER_NULL_ON_FAILURE) === null
-    						) {
+    						if (!in_array(strtolower($value), array('true', 'false'))) {
     							return 'invalid boolean';
     						}
     						break;
@@ -8332,13 +8430,19 @@ namespace Tqdev\PhpCrudApi\Middleware {
     						}
     						break;
     					case 'clob':
-    						// no checks needed
+    					case 'varchar':
+    						if ($column->hasLength() && mb_strlen($value, 'UTF-8') > $column->getLength()) {
+    							return 'string too long';
+    						}
     						break;
     					case 'blob':
     					case 'varbinary':
     						if (base64_decode($value, true) === false) {
     							return 'invalid base64';
     						}
+    						if ($column->hasLength() && strlen(base64_decode($value)) > $column->getLength()) {
+    							return 'string too long';
+    						}
     						break;
     					case 'geometry':
     						// no checks yet
@@ -8352,7 +8456,6 @@ namespace Tqdev\PhpCrudApi\Middleware {
     							return 'invalid integer';
     						}
     						break;
-    					case 'decimal':
     					case 'float':
     					case 'double':
     						if (!is_float($value) && !is_int($value)) {
@@ -8360,7 +8463,7 @@ namespace Tqdev\PhpCrudApi\Middleware {
     						}
     						break;
     					case 'boolean':
-    						if (!(is_int($value) && ($value === 1 || $value === 0)) && !is_bool($value)) {
+    						if (!is_bool($value) && ($value !== 0) && ($value !== 1)) {
     							return 'invalid boolean';
     						}
     						break;
@@ -8376,31 +8479,6 @@ namespace Tqdev\PhpCrudApi\Middleware {
     						return 'invalid integer';
     					}
     					break;
-    				case 'decimal':
-    					$value = "$value";
-    					if (strpos($value, '.') !== false) {
-    						list($whole, $decimals) = explode('.', $value, 2);
-    					} else {
-    						list($whole, $decimals) = array($value, '');
-    					}
-    					if (strlen($whole) > 0 && !ctype_digit($whole)) {
-    						return 'invalid decimal';
-    					}
-    					if (strlen($decimals) > 0 && !ctype_digit($decimals)) {
-    						return 'invalid decimal';
-    					}
-    					if (strlen($whole) > $column->getPrecision() - $column->getScale()) {
-    						return 'decimal too large';
-    					}
-    					if (strlen($decimals) > $column->getScale()) {
-    						return 'decimal too precise';
-    					}
-    					break;
-    				case 'varbinary':
-    					if (strlen(base64_decode($value)) > $column->getLength()) {
-    						return 'string too long';
-    					}
-    					break;
     			}
     		}
     		return (true);

docker/ubuntu20/run.sh

@@ -1,6 +1,6 @@
 #!/bin/bash
 echo "================================================"
-echo " Ubuntu 20.04 (PHP 7.3)"
+echo " Ubuntu 20.04 (PHP 7.4)"
 echo "================================================"
 
 echo -n "[1/4] Starting MySQL 8.0 ........ "

src/Tqdev/PhpCrudApi/Middleware/SanitationMiddleware.php

@@ -6,6 +6,7 @@ use Psr\Http\Message\ResponseInterface;
 use Psr\Http\Message\ServerRequestInterface;
 use Psr\Http\Server\RequestHandlerInterface;
 use Tqdev\PhpCrudApi\Column\Reflection\ReflectedTable;
+use Tqdev\PhpCrudApi\Column\Reflection\ReflectedColumn;
 use Tqdev\PhpCrudApi\Column\ReflectionService;
 use Tqdev\PhpCrudApi\Controller\Responder;
 use Tqdev\PhpCrudApi\Middleware\Base\Middleware;
@@ -30,11 +31,97 @@ class SanitationMiddleware extends Middleware
             if ($table->hasColumn($columnName)) {
                 $column = $table->getColumn($columnName);
                 $value = call_user_func($handler, $operation, $tableName, $column->serialize(), $value);
+                $value = $this->sanitizeType($table, $column, $value);
             }
         }
         return (object) $context;
     }
 
+    private function sanitizeType(ReflectedTable $table, ReflectedColumn $column, $value)
+    {
+        $tables = $this->getArrayProperty('tables', 'all');
+        $types = $this->getArrayProperty('types', 'all');
+        if (
+            (in_array('all', $tables) || in_array($table->getName(), $tables)) &&
+            (in_array('all', $types) || in_array($column->getType(), $types))
+        ) {
+            if (is_null($value)) {
+                return $value;
+            }
+            if (is_string($value)) {
+                $newValue = null;
+                switch ($column->getType()) {
+                    case 'integer':
+                    case 'bigint':
+                        $newValue = filter_var(trim($value), FILTER_VALIDATE_INT, FILTER_NULL_ON_FAILURE);
+                        break;
+                    case 'decimal':
+                        $newValue = filter_var(trim($value), FILTER_VALIDATE_FLOAT, FILTER_NULL_ON_FAILURE);
+                        if (is_float($newValue)) {
+                            $newValue = number_format($newValue, $column->getScale(), '.', '');
+                        }
+                        break;
+                    case 'float':
+                    case 'double':
+                        $newValue = filter_var(trim($value), FILTER_VALIDATE_FLOAT, FILTER_NULL_ON_FAILURE);
+                        break;
+                    case 'boolean':
+                        $newValue = filter_var(trim($value), FILTER_VALIDATE_BOOLEAN, FILTER_NULL_ON_FAILURE);
+                        break;
+                    case 'date':
+                        $time = strtotime(trim($value));
+                        if ($time !== false) {
+                            $newValue = date('Y-m-d', $time);
+                        }
+                        break;
+                    case 'time':
+                        $time = strtotime(trim($value));
+                        if ($time !== false) {
+                            $newValue = date('H:i:s', $time);
+                        }
+                        break;
+                    case 'timestamp':
+                        $time = strtotime(trim($value));
+                        if ($time !== false) {
+                            $newValue = date('Y-m-d H:i:s', $time);
+                        }
+                        break;
+                    case 'blob':
+                    case 'varbinary':
+                        // allow base64url format
+                        $newValue = strtr(trim($value), '-_', '+/');
+                        break;
+                    case 'clob':
+                    case 'varchar':
+                        $newValue = $value;
+                        break;
+                    case 'geometry':
+                        $newValue = trim($value);
+                        break;
+                }
+                if (!is_null($newValue)) {
+                    $value = $newValue;
+                }
+            } else {
+                switch ($column->getType()) {
+                    case 'integer':
+                    case 'bigint':
+                        if (is_float($value)) {
+                            $value = (int) round($value);
+                        }
+                        break;
+                    case 'decimal':
+                        if (is_float($value) || is_int($value)) {
+                            $value = number_format((float) $value, $column->getScale(), '.', '');
+                        }
+                        break;
+                }
+            }
+            // post process
+        }
+        return $value;
+    }
+
     public function process(ServerRequestInterface $request, RequestHandlerInterface $next): ResponseInterface
     {
         $operation = RequestUtils::getOperation($request);

src/Tqdev/PhpCrudApi/Middleware/ValidationMiddleware.php

@@ -34,7 +34,7 @@ class ValidationMiddleware extends Middleware
 				$column = $table->getColumn($columnName);
 				$valid = call_user_func($handler, $operation, $tableName, $column->serialize(), $value, $context);
 				if ($valid === true || $valid === '') {
-					$valid = $this->validateType($column, $value);
+					$valid = $this->validateType($table, $column, $value);
 				}
 				if ($valid !== true && $valid !== '') {
 					$details[$columnName] = $valid;
@@ -47,10 +47,14 @@ class ValidationMiddleware extends Middleware
 		return null;
 	}
 
-	private function validateType(ReflectedColumn $column, $value)
+	private function validateType(ReflectedTable $table, ReflectedColumn $column, $value)
 	{
+		$tables = $this->getArrayProperty('tables', 'all');
 		$types = $this->getArrayProperty('types', 'all');
-		if (in_array('all', $types) || in_array($column->getType(), $types)) {
+		if (
+			(in_array('all', $tables) || in_array($table->getName(), $tables)) &&
+			(in_array('all', $types) || in_array($column->getType(), $types))
+		) {
 			if (is_null($value)) {
 				return ($column->getNullable() ? true : "cannot be null");
 			}
@@ -77,15 +81,24 @@ class ValidationMiddleware extends Middleware
 							return 'invalid integer';
 						}
 						break;
-					case 'varchar':
-						if (mb_strlen($value, 'UTF-8') > $column->getLength()) {
-							return 'string too long';
-						}
-						break;
 					case 'decimal':
-						if (!is_numeric($value)) {
+						if (strpos($value, '.') !== false) {
+							list($whole, $decimals) = explode('.', $value, 2);
+						} else {
+							list($whole, $decimals) = array($value, '');
+						}
+						if (strlen($whole) > 0 && !ctype_digit($whole)) {
 							return 'invalid decimal';
 						}
+						if (strlen($decimals) > 0 && !ctype_digit($decimals)) {
+							return 'invalid decimal';
+						}
+						if (strlen($whole) > $column->getPrecision() - $column->getScale()) {
+							return 'decimal too large';
+						}
+						if (strlen($decimals) > $column->getScale()) {
+							return 'decimal too precise';
+						}
 						break;
 					case 'float':
 					case 'double':
@@ -97,9 +110,7 @@ class ValidationMiddleware extends Middleware
 						}
 						break;
 					case 'boolean':
-						if (
-							filter_var($value, FILTER_VALIDATE_BOOLEAN, FILTER_NULL_ON_FAILURE) === null
-						) {
+						if (!in_array(strtolower($value), array('true', 'false'))) {
 							return 'invalid boolean';
 						}
 						break;
@@ -119,13 +130,19 @@ class ValidationMiddleware extends Middleware
 						}
 						break;
 					case 'clob':
-						// no checks needed
+					case 'varchar':
+						if ($column->hasLength() && mb_strlen($value, 'UTF-8') > $column->getLength()) {
+							return 'string too long';
+						}
 						break;
 					case 'blob':
 					case 'varbinary':
 						if (base64_decode($value, true) === false) {
 							return 'invalid base64';
 						}
+						if ($column->hasLength() && strlen(base64_decode($value)) > $column->getLength()) {
+							return 'string too long';
+						}
 						break;
 					case 'geometry':
 						// no checks yet
@@ -139,7 +156,6 @@ class ValidationMiddleware extends Middleware
 							return 'invalid integer';
 						}
 						break;
-					case 'decimal':
 					case 'float':
 					case 'double':
 						if (!is_float($value) && !is_int($value)) {
@@ -147,7 +163,7 @@ class ValidationMiddleware extends Middleware
 						}
 						break;
 					case 'boolean':
-						if (!(is_int($value) && ($value === 1 || $value === 0)) && !is_bool($value)) {
+						if (!is_bool($value) && ($value !== 0) && ($value !== 1)) {
 							return 'invalid boolean';
 						}
 						break;
@@ -163,31 +179,6 @@ class ValidationMiddleware extends Middleware
 						return 'invalid integer';
 					}
 					break;
-				case 'decimal':
-					$value = "$value";
-					if (strpos($value, '.') !== false) {
-						list($whole, $decimals) = explode('.', $value, 2);
-					} else {
-						list($whole, $decimals) = array($value, '');
-					}
-					if (strlen($whole) > 0 && !ctype_digit($whole)) {
-						return 'invalid decimal';
-					}
-					if (strlen($decimals) > 0 && !ctype_digit($decimals)) {
-						return 'invalid decimal';
-					}
-					if (strlen($whole) > $column->getPrecision() - $column->getScale()) {
-						return 'decimal too large';
-					}
-					if (strlen($decimals) > $column->getScale()) {
-						return 'decimal too precise';
-					}
-					break;
-				case 'varbinary':
-					if (strlen(base64_decode($value)) > $column->getLength()) {
-						return 'string too long';
-					}
-					break;
 			}
 		}
 		return (true);

tests/config/base.php

@@ -4,7 +4,7 @@ $settings = [
     'username' => 'incorrect_username',
     'password' => 'incorrect_password',
     'controllers' => 'records,columns,cache,openapi,geojson',
-    'middlewares' => 'cors,reconnect,dbAuth,jwtAuth,basicAuth,authorization,validation,ipAddress,sanitation,multiTenancy,pageLimits,joinLimits,customization',
+    'middlewares' => 'cors,reconnect,dbAuth,jwtAuth,basicAuth,authorization,sanitation,validation,ipAddress,multiTenancy,pageLimits,joinLimits,customization',
     'dbAuth.mode' => 'optional',
     'dbAuth.returnedColumns' => 'id,username,password',
     'jwtAuth.mode' => 'optional',
@@ -35,6 +35,7 @@ $settings = [
     'sanitation.handler' => function ($operation, $tableName, $column, $value) {
         return is_string($value) ? strip_tags($value) : $value;
     },
+    'sanitation.tables' => 'forgiving',
     'validation.handler' => function ($operation, $tableName, $column, $value, $context) {
         return ($column['name'] == 'post_id' && !is_numeric($value)) ? 'must be numeric' : true;
     },

tests/functional/003_columns/015_update_types_table.log

@@ -12,7 +12,7 @@ true
 POST /records/types
 Content-Type: application/json
 
-{"integer":2,"bigint":3,"varchar":"abc","decimal":1.23,"float":1,"double":23.45,"boolean":true,"date":"1970-01-01","time":"00:00:01","timestamp":"2001-02-03 04:05:06","clob":"a","blob":"YQ==","geometry":"POINT(1 2)"}
+{"integer":2,"bigint":3,"varchar":"abc","decimal":"1.23","float":1,"double":23.45,"boolean":true,"date":"1970-01-01","time":"00:00:01","timestamp":"2001-02-03 04:05:06","clob":"a","blob":"YQ==","geometry":"POINT(1 2)"}
 ===
 200
 Content-Type: application/json
@@ -62,7 +62,7 @@ Content-Length: 101
 ===
 PUT /records/types/1
 
-{"integer":"12345678901"}
+{"integer":"2.3"}
 ===
 422
 Content-Type: application/json
@@ -72,57 +72,57 @@ Content-Length: 101
 ===
 PUT /records/types/1
 
-{"bigint":"12345678901234567890"}
+{"integer":2.3}
 ===
 422
 Content-Type: application/json
-Content-Length: 100
+Content-Length: 101
 
-{"code":1013,"message":"Input validation failed for 'types'","details":{"bigint":"invalid integer"}}
+{"code":1013,"message":"Input validation failed for 'types'","details":{"integer":"invalid integer"}}
 ===
 PUT /records/types/1
 
-{"varchar":"12345678901"}
+{"integer":"12345678901"}
 ===
 422
 Content-Type: application/json
 Content-Length: 101
 
-{"code":1013,"message":"Input validation failed for 'types'","details":{"varchar":"string too long"}}
+{"code":1013,"message":"Input validation failed for 'types'","details":{"integer":"invalid integer"}}
 ===
 PUT /records/types/1
 
-{"decimal":"12.23.34"}
+{"bigint":"12345678901234567890"}
 ===
 422
 Content-Type: application/json
-Content-Length: 101
+Content-Length: 100
 
-{"code":1013,"message":"Input validation failed for 'types'","details":{"decimal":"invalid decimal"}}
+{"code":1013,"message":"Input validation failed for 'types'","details":{"bigint":"invalid integer"}}
 ===
 PUT /records/types/1
 
-{"decimal":1131313145345}
+{"varchar":"12345678901"}
 ===
 422
 Content-Type: application/json
-Content-Length: 103
+Content-Length: 101
 
-{"code":1013,"message":"Input validation failed for 'types'","details":{"decimal":"decimal too large"}}
+{"code":1013,"message":"Input validation failed for 'types'","details":{"varchar":"string too long"}}
 ===
 PUT /records/types/1
 
-{"decimal":"1234567.123"}
+{"decimal":"12.23.34"}
 ===
 422
 Content-Type: application/json
-Content-Length: 103
+Content-Length: 101
 
-{"code":1013,"message":"Input validation failed for 'types'","details":{"decimal":"decimal too large"}}
+{"code":1013,"message":"Input validation failed for 'types'","details":{"decimal":"invalid decimal"}}
 ===
 PUT /records/types/1
 
-{"decimal":11313131e+5}
+{"decimal":"1131313145345"}
 ===
 422
 Content-Type: application/json
@@ -132,17 +132,17 @@ Content-Length: 103
 ===
 PUT /records/types/1
 
-{"decimal":"123456.12345"}
+{"decimal":"1234567.123"}
 ===
 422
 Content-Type: application/json
-Content-Length: 105
+Content-Length: 103
 
-{"code":1013,"message":"Input validation failed for 'types'","details":{"decimal":"decimal too precise"}}
+{"code":1013,"message":"Input validation failed for 'types'","details":{"decimal":"decimal too large"}}
 ===
 PUT /records/types/1
 
-{"decimal":113131.3145345}
+{"decimal":"123456.12345"}
 ===
 422
 Content-Type: application/json
@@ -152,7 +152,7 @@ Content-Length: 105
 ===
 PUT /records/types/1
 
-{"decimal":11313131E-5}
+{"decimal":"113131.3145345"}
 ===
 422
 Content-Type: application/json

tests/functional/003_columns/016_update_forgiving_table.log

@@ -0,0 +1,383 @@
+===
+POST /columns
+
+{"name":"forgiving","type":"table","columns":[{"name":"id","type":"integer","pk":true},{"name":"integer","type":"integer"},{"name":"bigint","type":"bigint"},{"name":"varchar","type":"varchar","length":10},{"name":"decimal","type":"decimal","precision":10,"scale":4},{"name":"float","type":"float"},{"name":"double","type":"double"},{"name":"boolean","type":"boolean"},{"name":"date","type":"date"},{"name":"time","type":"time"},{"name":"timestamp","type":"timestamp"},{"name":"clob","type":"clob"},{"name":"blob","type":"blob"},{"name":"geometry","type":"geometry"}]}
+===
+200
+Content-Type: application/json
+Content-Length: 4
+
+true
+===
+POST /records/forgiving
+Content-Type: application/json
+
+{"integer":2,"bigint":3,"varchar":"abc","decimal":1.23,"float":1,"double":23.45,"boolean":true,"date":"1970-01-01","time":"00:00:01","timestamp":"2001-02-03 04:05:06","clob":"a","blob":"YQ==","geometry":"POINT(1 2)"}
+===
+200
+Content-Type: application/json
+Content-Length: 1
+
+1
+===
+PUT /records/forgiving/1
+
+{"boolean":"true"}
+===
+200
+Content-Type: application/json
+Content-Length: 1
+
+1
+===
+GET /records/forgiving/1?include=boolean
+===
+200
+Content-Type: application/json
+Content-Length: 16
+
+{"boolean":true}
+===
+PUT /records/forgiving/1
+
+{"boolean":"yes"}
+===
+200
+Content-Type: application/json
+Content-Length: 1
+
+1
+===
+GET /records/forgiving/1?include=boolean
+===
+200
+Content-Type: application/json
+Content-Length: 16
+
+{"boolean":true}
+===
+PUT /records/forgiving/1
+
+{"boolean":"1"}
+===
+200
+Content-Type: application/json
+Content-Length: 1
+
+1
+===
+GET /records/forgiving/1?include=boolean
+===
+200
+Content-Type: application/json
+Content-Length: 16
+
+{"boolean":true}
+===
+PUT /records/forgiving/1
+
+{"boolean":1}
+===
+200
+Content-Type: application/json
+Content-Length: 1
+
+1
+===
+GET /records/forgiving/1?include=boolean
+===
+200
+Content-Type: application/json
+Content-Length: 16
+
+{"boolean":true}
+===
+PUT /records/forgiving/1
+
+{"integer":" 2\n"}
+===
+200
+Content-Type: application/json
+Content-Length: 1
+
+1
+===
+GET /records/forgiving/1?include=integer
+===
+200
+Content-Type: application/json
+Content-Length: 13
+
+{"integer":2}
+===
+PUT /records/forgiving/1
+
+integer=2%20
+===
+200
+Content-Type: application/json
+Content-Length: 1
+
+1
+===
+GET /records/forgiving/1?include=integer
+===
+200
+Content-Type: application/json
+Content-Length: 13
+
+{"integer":2}
+===
+PUT /records/forgiving/1
+
+{"integer":1.99999}
+===
+200
+Content-Type: application/json
+Content-Length: 1
+
+1
+===
+GET /records/forgiving/1?include=integer
+===
+200
+Content-Type: application/json
+Content-Length: 13
+
+{"integer":2}
+===
+PUT /records/forgiving/1
+
+{"bigint":" 3\n"}
+===
+200
+Content-Type: application/json
+Content-Length: 1
+
+1
+===
+GET /records/forgiving/1?include=bigint
+===
+200
+Content-Type: application/json
+Content-Length: 12
+
+{"bigint":3}
+===
+PUT /records/forgiving/1
+
+{"bigint":2.99999}
+===
+200
+Content-Type: application/json
+Content-Length: 1
+
+1
+===
+GET /records/forgiving/1?include=bigint
+===
+200
+Content-Type: application/json
+Content-Length: 12
+
+{"bigint":3}
+===
+PUT /records/forgiving/1
+
+{"decimal":"1.23"}
+===
+200
+Content-Type: application/json
+Content-Length: 1
+
+1
+===
+GET /records/forgiving/1?include=decimal
+===
+200
+Content-Type: application/json
+Content-Length: 20
+
+{"decimal":"1.2300"}
+===
+PUT /records/forgiving/1
+
+{"decimal":"1.23004"}
+===
+200
+Content-Type: application/json
+Content-Length: 1
+
+1
+===
+GET /records/forgiving/1?include=decimal
+===
+200
+Content-Type: application/json
+Content-Length: 20
+
+{"decimal":"1.2300"}
+===
+PUT /records/forgiving/1
+
+{"decimal":"1.23006"}
+===
+200
+Content-Type: application/json
+Content-Length: 1
+
+1
+===
+GET /records/forgiving/1?include=decimal
+===
+200
+Content-Type: application/json
+Content-Length: 20
+
+{"decimal":"1.2301"}
+===
+PUT /records/forgiving/1
+
+float=1%20
+===
+200
+Content-Type: application/json
+Content-Length: 1
+
+1
+===
+GET /records/forgiving/1?include=float
+===
+200
+Content-Type: application/json
+Content-Length: 11
+
+{"float":1}
+===
+PUT /records/forgiving/1
+
+{"double":" 23.45e-1 "}
+===
+200
+Content-Type: application/json
+Content-Length: 1
+
+1
+===
+GET /records/forgiving/1?include=double
+===
+200
+Content-Type: application/json
+Content-Length: 16
+
+{"double":2.345}
+===
+PUT /records/forgiving/1
+
+{"date":"2020-12-05"}
+===
+200
+Content-Type: application/json
+Content-Length: 1
+
+1
+===
+GET /records/forgiving/1?include=date
+===
+200
+Content-Type: application/json
+Content-Length: 21
+
+{"date":"2020-12-05"}
+===
+PUT /records/forgiving/1
+
+{"date":"December 20th, 2020"}
+===
+200
+Content-Type: application/json
+Content-Length: 1
+
+1
+===
+GET /records/forgiving/1?include=date
+===
+200
+Content-Type: application/json
+Content-Length: 21
+
+{"date":"2020-12-20"}
+===
+PUT /records/forgiving/1
+
+{"time":"13:15"}
+===
+200
+Content-Type: application/json
+Content-Length: 1
+
+1
+===
+GET /records/forgiving/1?include=time
+===
+200
+Content-Type: application/json
+Content-Length: 19
+
+{"time":"13:15:00"}
+===
+PUT /records/forgiving/1
+
+{"timestamp":"2012-1-1 23:46"}
+===
+200
+Content-Type: application/json
+Content-Length: 1
+
+1
+===
+GET /records/forgiving/1?include=timestamp
+===
+200
+Content-Type: application/json
+Content-Length: 35
+
+{"timestamp":"2012-01-01 23:46:00"}
+===
+PUT /records/forgiving/1
+
+{"clob":"𠜎𠜱𠝹𠱓𠱸𠲖𠳏𠳕𠴕𠵼𠵿𠸎𠸏𠹷𠺝𠺢𠻗"}
+===
+200
+Content-Type: application/json
+Content-Length: 1
+
+1
+===
+PUT /records/forgiving/1
+
+{"blob":"!"}
+===
+422
+Content-Type: application/json
+Content-Length: 101
+
+{"code":1013,"message":"Input validation failed for 'forgiving'","details":{"blob":"invalid base64"}}
+===
+PUT /records/forgiving/1
+
+{"blob":"T8O5IGVzdCBsZSBjYWbDqSBsZSBwbHVzIHByb2NoZT8"}
+===
+200
+Content-Type: application/json
+Content-Length: 1
+
+1
+===
+DELETE /columns/forgiving
+===
+200
+Content-Type: application/json
+Content-Length: 4
+
+true