[cs] Added option to exclude C# using directives from CPD analysis

[tiobe]

Some of our customers complained that their duplicated code was to high because because a lot of files have duplicate using directives. For example:

using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Xml.Serialization;
using CommandLine;
using CommandLine.Text;

namespace TIOBE.TICS.CIL.Static.CmdLine

This duplicates cannot be prevented, because each file needs to specify its dependencies, so the 'using' directives should be ignored during duplicated code analysis. To fix this I added a option called 'ignore_usings' to the CPD CS tokenizer, which can be added by specifying the following system property on the CPD command line: '-Dignore_usings=true'.

As you can see in the diff, the 'using' directive tokens are not omitted from the token stream but replaced by a random token. This is done to prevent that we get duplication blocks that spans over the using directives. (Starts before and end afterwards.)

As you can see this pull request also contains an older commit of us. ('Extended Objective-C grammar to accept Unicode characters in identifiers' ) This commit was already merged in the 'pmd/5.4.x' branch, but unfortunately we cannot upgrade to PMD 5.4 because a lot of our customers still use Java 1.6.

If both commit are merged and released in PMD 5.3, we can use an official PMD release again instead of our own builds. Please let me know if there are changes needed before this pull request can be accepted.

[buildhive]

Andreas Dangel » pmd #341 SUCCESS
This pull request looks good
(what's this?)

[adangel]

Thanks for the PR.
I've extended it to have the new option "ignoreUsings" available as a commandline option, in the CPD Ant task and in the CPD GUI.