Last night I was having a security session on a community website. Some content was controlled by TypoScript and I did some penetration tests on Cross-Site Scripting (XSS).

The website allows anyone to register and enter data to be shown on the website. The threat was that a possible attacker could create an account and enter malicious data.

There were two possibilities to protect the website from being XSSed:

• 1. validate and filter the user data before it gets saved,
• 2. escape and filter the user data before it gets displayed on the website.

Adding XSS protection in TypoScript

Let's have a look at the second step where TypoScript comes into the match. For testing purpose I put the following malicious data into the DB fields:

<script>alert('fieldname')</script>

Whenever the data is not escaped before being displayed, an alert window pops up reporting the fieldname.

The website makes heavy use of dataWrap to show the data in the frontend. The cool thing with dataWrap is, that it allows to shrink your TypoScript code by putting multiple getText requests into a one liner. Have a look at the first example:

Example 1

page.10 = TEXT
page.10 {
  dataWrap = Hello {field:title} {field:firstname} {field:lastname}
}

The output of a regular dataset here would be: Hello Dr. Franken Stein. But when testing a malicious dataset with <script>alert('fieldname')</script> I ran into XSS with popups all around.

So the next step was to remove all bad code from the result set. I decided to use htmlSpecialChars:

Example 2

page.10 = TEXT
page.10 {
  dataWrap = Hello {field:title} {field:firstname} {field:lastname}
  htmlSpecialChars = 1
}

My expectation was that the complete string would be parsed through htmlspecialchars(). If there was malicious data in the name fields, it would get escaped. But testing the stuff revealed the same behaviour as in example 1: The field data was not escaped with htmlspecialchars and I ended up with javascript alerts popping around on my screen. Next, I tried a different approach:

Example 3

page.10 = TEXT
page.10.value = Hello
page.20 = TEXT
page.20 {
  field = title
  htmlSpecialChars = 1
}
page.30 {
  field = firstname
  htmlSpecialChars = 1
}
page.40 {
  field = lastname
  htmlSpecialChars = 1
}

This time, testing was successful: All HTML data was escaped and no popups: The result was again Hello Dr. Franken Stein and Hello <script>alert('title')</script> <script>alert('firstname')</script> <script>alert('lastname')</script> as plain text with escaped characters.

The drawback was that the code was bloated. I tried to shrink it by switching back from field to dataWrap. And this time I tried a different approach to apply htmlSpecialChars:

Example 4

page.10 = COA
page.10.stdWrap.htmlSpecialChars = 1
page.10.10 = TEXT
page.10.10.dataWrap = Hello {field:title} {field:firstname} {field:lastname}

... and Bingo: No popups, all fields were correctly parsed through htmlspecialchar().

Summary

Although example 2 appeared logical to me, it did not work. The reason was that the getText sections in dataWrap were not being touched by surrounding stdWrap functions. As example 4 showed, the solution was to use COA as a container for the data. That made it possible to wrap the complete string with htmlSpecialChars.

What's the lesson learned of this case?

Never trust your intuitive expectation of how code works when it comes to security issues. Better test twice than overlook a leak. Test driven development is the way to go.

There are several other websites hosted on the server and TYPO3 is not the only one eating performance. At peak periods, the server load is very high and pages are served with much delay. Not only visitors suffer from slow websites, but there are rumors that google may take performance into ranking account (so called speed rank).

Solution

Since this website is driven by TYPO3, I decided to install the static file cache extension (nc_static_file_cache).This extension puts pages from the page cache to files in the filesystem. Whenever a page is requested, it gets served from the file system without involving TYPO3.

Test results (1)

To test the performance gain, I used ApacheBench (ab) on the start page:

spock@enterprise:/tmp$ ab -c 10 -n 1000 http ://www.t3node.com/

The result was disappointing 8 req/s.

After some investigation I realized that most of my pages did not profit from this cache. The info module of the static file cache extension reported, that they could not be statically served because they contain uncached USER_INT objects. Inspections with the admpanel revealed two extensions which use USER_INT:

• 'News: most popular' extension (nc_ttnews_mostpopular), which was used to count and display hits of the blog articles.
• Twitter RSS feed extension (tp_twitter), which was used to show my latest three tweets.

Fixing the issues

I decided to remove the 'most popular' extension, since I never really liked the approach to save statistic data in the database table of the entity. I am going to look for something similar, maybe using some JavaScript counter or Apache access_log in combination with COA_GO.

The Tweet is still shown, but as a cached USER object which will be updated once a day. I think that's a good compromise, since my twitter activity is anyway not so high.

Test results (2)

To test the performance gain, I used ApacheBench (ab) on the start page:

spock@enterprise:/tmp$ ab -c 10 -n 1000 http ://www.t3node.com/

With USER_INT cObjects: 8 req/s.

Without USER_INT cObjects and without static file cache: 40 req/s.

Without USER_INT cObjects and with static file cache: 500 req/s.

That's 62.5 times faster than before! Ready to get slashdottet :-)

Conclusion

Performance can be crucial. If your website feels slow, search for bottlenecks like USER_INT extensions. If you can manage to get rid of USER_INT cObjects, chances are that you can also profit by a static file cache. That could push your website performance to lightning speed.

As I have already shown in an article have a year ago, TypoScript is not save against SQL injections. I introduced a case, where a numerical GET parameter (a uid as integer) is used in the WHERE clause of a TypoScript CONTENT cObject. But the parameter could be misused to inject malicious SQL into the WHERE clause. The solution was to parse the GET parameter through the stdWrap function intval. It ensured that only integer values were passed to the WHERE clause.

The posting you are now about to read is meant as a follow-up to that article. It demonstrates a more advanced case: using a string of a GET parameter inside an SQL query of CONTENT.

Example code

Imagine you offer some search function to your website visitors. The search term is send as a GET parameter and TypoScript compares the term with the header of the content elements of a page. All content element of a page which is matching this comparison are shown. The page content is:

We use the TypoScript cObject CONTENT to output all matching content elements of a page. The search term is inserted into the WHERE part of the SQL query:

page.10 = CONTENT
page.10 {
  table = tt_content
  select {
    pidInList = 35
    andWhere.cObject = TEXT
    andWhere.cObject {
      data = GPvar:tx_myext|header
      wrap = header=”|”    
    }
  }
} 

Result

If the URL contains the GET parameter tx_myext[header] it uses the given value in the SQL WHERE clause. For example, try this URL to get all content elements which use the term My products as header:

http ://www.example.com/index.php?id=123&tx_myext[header]=My products

That will result in a SQL clause like this:

SELECT * FROM tt_content ... WHERE ... AND header=”My products” ...

Have a look at the output of this TypoScript object. In the frontend, you'll see this:

SQL injection vulnerability

With the TypoScript above, your website is vulnerable to SQL injections. Imagine a visitor uses the following malicious URL for SQL injection:

http ://www.example.com/index.php?id=123&tx_myext[header]=abc" OR 1=1 OR 1="1

This will result in a SQL clause:

SELECT * FROM tt_content ... WHERE ... AND header=”abc” OR 1=1 OR 1=”1” ...

In the frontend, you'll see something like this:

The website shows all content elements, no matter if they are deleted or hidden!

Solution: quoting and escaping the string

To prevent bad guys from injecting SQL with the search term, you need to quote the string and escape possible control characters. Unfortunately, there's no function in stdWrap to do that.

I have recently released the "quote functions for stdWrap" extension (sm_stdwrapquote) to TER. This extension adds two functions to TypoScript stdWrap:

• quoteStr to escape special values
• fullQuoteStr to escape special values and add quotes

It implements the functionality of t3lib_DB->quoteStr() and t3lib_DB->fullQuoteStr() to stdWrap. This helps you sanitize and safely use GET/POST strings in TypoScript CONTENT objects without too much hassle.

Example code with protection against SQL injection

The extension provides a simple method to protect your TypoScript from SQL injection. You simply add fullQuoteStr to the andWhere part of your CONTENT object. It can be used like a regular function of stdWrap. When using fullQuoteStr, you also don't need quotes in the wrap:

page.10 = CONTENT
page.10 {
  table = tt_content
  select {
    pidInList = 35
    andWhere.cObject = TEXT
    andWhere.cObject {
      data = GPvar:tx_myext|header
      fullQuoteStr = tt_content
      wrap = header=|
    }
  }
} 

The result is that SQL injection will not work anymore. When using the malicious URL, the frontend won't show any content elements at all. fullQuoteStr escapes all quotes inside the GET parameter and quotes the whole string. Data can be safely used with fullQuoteStr inside an SQL SELECT query of CONTENT.

For some more information, read the manual which is shiped with the extension.

Conclusion

You always have to protect your SQL queries of TypoScript cObject CONTENT against SQL injection. Since stdWrap does not ship sufficient functions to sanitize user parameters, use the fullQuoteStr function of the extension sm_stdwrapquote.

Future implementation in TYPO3 core

There's an RFC in the TYPO3 core team list to implement these functions into the core. Unfortunately this did not happen with version 4.3. But I try to push this into 4.4. If you like it, go to the core list and vote for feature #12094.

Meanwhile I found out that ve_guestbook disables complete caching of pages with the guestbook form. I don't like that and it's not necessary. The plugin could be rendered as USER_INT instead of disabling page cache. I sent a bug report #12131 and appended a patch which fixes this issue: http://bugs.typo3.org/view.php?id=12131

Whenever you type some characters in the suggest field, the wizard searches for records matching the label field of given tables and immediately presents a list of results (find-as-you-type). Therefore it scans the table column which was set in the "label"-field or "label_alt"-field given in the $TCA['ctrl'] section of the tables. That's for example the title field in the pages table or the header field in tt_content. The suggest wizard can be added to a group or select field like any other wizard by adding it to the $TCA['columns']['fieldname']['config'] section. At the time of writing this article, it's supposed to work only for database records: that means fields of type "group" having set 'internal_type' => 'db'or fields of type "select" having 'foreign_table' => tablename. Files and static items don't work ATM.

How to add the suggest wizard to a field of your extension?

The suggest wizard is added to a field like any other wizard in TCA. If you have no idea of using wizards yet, have a look at wizard section of the TYPO3 Core API documentation.
For the suggest wizard, there are two places for configuration: The $TCA['columns']['fieldname']['config'] section and page TSconfig.

A simple scenario for demonstration

There's an extension appended in this article for demonstration. The extension ships two new tables for articles and authors. To set the author in the article table, you can select records from the author table. Therefore the author field of articles not only shows a list of selectable authors, but also implements the new suggest wizard.

Downloading the example extension

If you like to see the wizard in action, you can download the example extension "Demonstrate the TCA suggest wizard (demo_suggest)".

The extension ships some sample data as a t3d file inside res/suggest_demo_dataset.t3d which can be imported with the import/export tool. You need to move this file into the fileadmin/ directory to be able to import the data into your TYPO3 installation.

Implementing the wizard

The below example code shows how to add the wizard to the "author" field of the table "tx_demosuggest_article" in TCA. In the "allowed" section, you need to add the table which should be searched for records. (When editing an article, you want to search for authors).

TCA (tca.php)

$TCA['tx_demosuggest_article']['columns']['author']['config'] => array(
    'type' => 'select',    
    'foreign_table' => 'tx_demosuggest_author',
    'allowed' => 'tx_demosuggest_author',    
    'size' => 10,    
    'minitems' => 0,
    'maxitems' => 100,
    'wizards' => array(
        'suggest' => array(    
            'type' => 'suggest',
        ),
    ),
);

There are some configuration options which can be set directly in the TCA-wizard section or by using pageTS. The below example limits the number of results to 5 and searches only in page 123:

TCA (tca.php)

$TCA['tx_demosuggest_article']['columns']['author']['config'] => array(
    (...)
    'wizards' => array(
        'suggest' => array(    
            'type' => 'suggest',
            'tx_demosuggest_author' => array(
                'maxItemsInResultList' => 5,
                'pidList' => '123',
            ),
        ),
    ),
);

These options can also be set in page TSconfig:

pageTS (TSconfig)

TCEFORM.suggest.tx_demosuggest_author {
  maxItemsInResultList = 5
  pidList = 123
}

Instead of setting these options for a certain table, you can also set a default value for any table:

pageTS (TSconfig)

TCEFORM.suggest.default {
  maxItemsInResultList = 5
}

Find a full list of all options in the TYPO3 Core API documentation. As long as 4.3 is not realeased, please refer to the corresponding section of pending documentation in TYPO3 wiki.

Of course you can also use this wizard in any table of the TYPO3 core. It is already implemented in the storage_pid field of pages.

Further Reading

• TYPO3 Core API Documentation (once 4.3 is released)
• Wizards configuration in TYPO3 Core API Documentation
• Pending documentation in the TYPO3 wiki (until 4.3 is finally released)

Hooks in TYPO3

Using hooks is a common way to add functionality or change behaviour of the TYPO3 core. Although it sometimes looks like hardcore coding stuff, it's not that hard to understand. The TYPO3 Core API documentation on typo3.org gives you an overview about hooks in TYPO3. To get a list of hooks which exist in TYPO3, you can use the extension dmc HookList (dmc_hooklist).

Preassumptions

This tutorial assumes you have an extension called extExample. extExample is used as the extension key. The extension stores its records in the DB table tx_extExample_data. The records are kept in a sysfolder in the BE. There's a frontend plugin tx_extExample_pi1 which displays a single record on a page (page id=123) and using the GET-parameter tx_extExample_pi1[showUid] for the uid of the record. A very common type of frontend extension, isn't it? Just replace the emphasized keywords to use the snippets in your extension.

What do you need to do in your own extension to get this button running?

There's only three things to do:

• add a class to your extension with a method to set the proper preview parameters,
• register the hook in your extension to tell TYPO3 to call this method,
• set the page ID of the single view in pageTS.

Writing a method to set the preview parameters

The method adds the needed information to a global variable which itself is processed by TCE when openening the popup window for the preview.

typo3conf/ext/extExample/classes/class.tx_extExample_tcemain.php
<?php
class tx_extExample_tcemain {

  public function processDatamap_preProcessFieldArray(&$fieldArray, $table, $uid, &$pObj) {
 
   if ($table == 'tx_extExample_data' && isset($GLOBALS['_POST']['_savedokview_x']) && !$GLOBALS['BE_USER']->workspace) {

        // The GETparam which is used in the single view of the plugin
        $getParam = 'tx_extExample_pi1[showUid]';

        // get page TSconfig
        $pageTS = t3lib_BEfunc::getPagesTSconfig($GLOBALS['_POST']['popViewId']);

        // get pid of single view
        $singlePid = $pageTS['tx_extExample.']['singlePid'];

        if ($singlePid) {

          // set page id to be shown
          $GLOBALS['_POST']['popViewId'] = $singlePid;

          // don't cache the page
          $GLOBALS['_POST']['popViewId_addParams'] = '&no_cache=1';

          // set language parameter
          if ($fieldArray['sys_language_uid'] > 0) {
            $GLOBALS['_POST']['popViewId_addParams'] .= '&L=' . $fieldArray['sys_language_uid'];
          }

          // set plugin GETparams
          $GLOBALS['_POST']['popViewId_addParams'] .= '&' . $getParam . '=' . $uid;
        }
      }
    }
  }
}

if (defined('TYPO3_MODE') && $TYPO3_CONF_VARS[TYPO3_MODE]['XCLASS']['ext/extExample/classes/class.tx_extExample_tcemain.php']) {
  include_once($TYPO3_CONF_VARS[TYPO3_MODE]['XCLASS']['ext/extExample/classes/class.tx_extExample_tcemain.php']);
}

?>

Registering the hook

Hooks are registered in your extension by adding a simple oneliner to ext_localconf.php:

ext_localconf.php
$GLOBALS ['TYPO3_CONF_VARS']['SC_OPTIONS']['t3lib/class.t3lib_tcemain.php']['processDatamapClass'][] = 'EXT:exampleExt/classes/class.tx_exampleExt_tcemain.php:tx_exampleExt_tcemain';

Set the singlePid in pageTS

Finaly, you need to set the page ID of the frontend single view in pageTS. It's the pid of the page which should be opened in the preview.

pageTS (TSconfig)
tx_extExample.singlePid = 123

That's it! Now clear the configuration cache in the TYPO3 backend, open a record and try to "save and preview".

This solution works only in LIVE workspace. I didn't test it within otther workspaces.

To get an idea what this article is exactly about, go login to a TYPO3 backend, open a content element of type "Text" and change its type to "Image". The form of the content element will reload with some new fields appearing and others disappearing.

This is what I call here dynamic forms. A single content element can be used for different tasks using different form fields. Each form type contains an individual set of fields.

TCA configuration to get dynamic forms

In TCA, the above example is achieved by two settings:

• reloading the form after changing the "type" field (dynamic form)
• configuring the field sets for each type (types).

The first setting is configured in TCA['ctrl']['type'] section by setting a certain table field to act as a form type switcher. In tt_content for example it look like this:

TCA
'ctrl' => array(
    'type' => 'CType',
),

The second setting is arranged in TCA['ctrl']['types'] section by setting all individual fields for any type of form. For example:

TCA
'types' => array(
    '1' => array(
        'showitem' => 'CType, header, bodytext'
    ),
    '2' => array(
        'showitem' => 'CType, header, image'
    ),
),

In the above example we get a form with two types. One type is showing the type, a header and a text field, the other shows an image field instead of the text field. The field CType is used to switch between the types.

FlexForms configuration to get dynamic forms

In FlexForms we have to use a slightly different approach:

• using <onChange>reload</onChange> instead of TCA[ctrl']['type'] to reload forms after changing the type field (onChange)
• adding a condition to all FlexForm fields which expects a certain value of the type field. (displayCondition)

Our example form would look like this:

FlexForm
<myType>
  <TCEforms>
    <label>Type of form</label>
    <onChange>reload</onChange>
    <config>
      <type>select</type>
      <size>1</size>
      <maxitems>1</maxitems>
      <items>
        <numIndex index="0" type="array">
          <numIndex index="0"></numIndex>
          <numIndex index="1">0</numIndex>
        </numIndex>
        <numIndex index="1" type="array">
          <numIndex index="0">Type No. 1 for texts</numIndex>
          <numIndex index="1">text</numIndex>
        </numIndex>
        <numIndex index="2" type="array">
          <numIndex index="0">Type No. 2 for images</numIndex>
          <numIndex index="1">image</numIndex>
        </numIndex>
      </items>
    </config>
  </TCEforms>
</myType>
<bodytext>
  <TCEforms>
    <label>Your Text</label>
    <displayCond>FIELD:myType:=:text</displayCond>
    <config>
      <type>input</type>
    </config>
  </TCEforms>
</bodytext>
<image>
  <TCEforms>
    <label>Your Image</label>
    <displayCond>FIELD:myType:=:image</displayCond>
    <config>
      <type>input</type>
    </config>
  </TCEforms>
</image>

Explanation of the FlexForm displayCond feature

The displayCond feature is explained in the ['columns'][fieldname] section of the TYPO3 core API documentation.

In our case we use FIELD:myType:IN:value where value is a certain value of the type field (text or image):

FIELD:myType:=:text

The field text is displayed only if the field myType is set to the value "text".

FlexForms documentation unleashed?

FlexForms are almost as powerful as regular TCA forms. The configuration of FlexForms is slightly different than TCA: Most of the features use equivalent syntax, some have their own. Unfortunately, the documenation of FlexForms in most cases refers to TCA, instead of being discussed in a separate chapter - resulting in very few FlexForm examples.

You need to pass the 36 chambers of TCA to reveal the secret knowledge of FlexForms. Have patience and come up with some good keywords when search the mailing lists and forums for FlexForms examples.

The problem was to have dynamic values in the frames (section_frame) field of content elements, depending on the value of the column (colPos) field. If normal column was choosen, frames should offer value 80 and 81. If right column was choosen, frames should offer 82 and 83.

The first thing I did was to add colPos to the list of fields, which trigger saving and reloading the content element on change. I kickstarted a new extension and added these lines to extTables.php:

$updateFields = t3lib_div::trimExplode(
  ',',
  $TCA['tt_content']['ctrl']['requestUpdate']
);
if (in_array('colPos', $updateFields)) {
  $TCA['tt_content']['ctrl']['requestUpdate'] .= ',colPos';
} 

Then I changed the TCA of section_frame to use itemsProcFunc to customize its items:

$TCA['tt_content']['columns']['section_frame']['config']['items'] = array();
$TCA['tt_content']['columns']['section_frame']['config']['itemsProcFunc'] = 'tx_myext_setsectionframeitems->main';
if (TYPO3_MODE == 'BE') {
  include_once(t3lib_extMgm::extPath($_EXTKEY).'class.tx_myext_setsectionframeitems.php');
} 

Next step was to implement the class, which controls the items. The class itself was stored in a seperate file called class.tx_myext_setsectionframeitems.php:

class tx_myext_setsectionframeitems {
  function main(&$params, &$pObj) {
    global $LANG;
    $selectOptions = array();

   // Allowed fields for normal column
    $normalFields = array(
      $LANG->sL("LLL:EXT:myext/locallang_tca.xml:tt_content.section_frame.I.80") => 80,
      $LANG->sL("LLL:EXT:myext/locallang_tca.xml:tt_content.section_frame.I.81") => 81
     );
    // Allowed fields for right column
    $rightFields = array(
      $LANG->sL("LLL:EXT:myext/locallang_tca.xml:tt_content.section_frame.I.82") => 82,
      $LANG->sL("LLL:EXT:myext/locallang_tca.xml:tt_content.section_frame.I.83") => 83
    );
    // Get value of tt_content.colPos
    $key = key($pObj->cachedTSconfig);
    $colPos = $pObj->cachedTSconfig[$key]['_THIS_ROW']['colPos'];
    // Add items
    if ($colPos == 2) {
      $selectOptions = $rightFields;
    } else {
      $selectOptions = $normalFields;
    }
    foreach ($selectOptions as $key => $value) {
      $params['items'][] = array(
        $key,
        $value
      );
    }
  }
}

if (defined('TYPO3_MODE') && $TYPO3_CONF_VARS[TYPO3_MODE]['XCLASS']['ext/myext/class.tx_myext_setsectionframeitems.php'])  {
  include_once($TYPO3_CONF_VARS[TYPO3_MODE]['XCLASS']['ext/myext/class.tx_myext_setsectionframeitems.php']);
} 

The custom labels were stored in a usual locallang file.

Finally, I did some TSconfig tuning in extTables.php to prevent invalid field values:

t3lib_extMgm::addPageTSConfig('
   TCEFORM.tt_content.section_frame.disableNoMatchingValueElement = 1
'); 

That's it. The whole stuff looks a bit ugly to me. So if you know a smarter solution, I'd be glad if you leave a comment.

I already got in contact with FLOW3 at T3DD09, listening to several talks about its architecture and components and playing a bit with the code. This milestone release comes just in time with Extbase, the forthcoming backport of the MVC pattern of FLOW3 for TYPO3 version 4.x.

I am very curious about the progress of these projects, since I am already infected by the new design patterns and programming paradigms. I also see more and more people testing and writing FLOW3 and Extbase stuff which makes me confident about its success. Still there is a lot of criticism and scepticism about FLOW3, especially the question of performance. Anyway this is a first alpha release and there's still enough time and code left to solve these problems.

Keep on with your good work!