2
0
mirror of https://expo.survex.com/repositories/troggle/.git synced 2024-11-25 16:51:54 +00:00

[svn] horrid .svns copied accidentally

This commit is contained in:
goatchurch 2009-06-28 21:26:35 +01:00
parent db5e315db0
commit 16b7404d9b
45 changed files with 8750 additions and 0 deletions

View File

@ -0,0 +1,23 @@
Copyright (c) 2007-2009 Marijn Haverbeke
This software is provided 'as-is', without any express or implied
warranty. In no event will the authors be held liable for any
damages arising from the use of this software.
Permission is granted to anyone to use this software for any
purpose, including commercial applications, and to alter it and
redistribute it freely, subject to the following restrictions:
1. The origin of this software must not be misrepresented; you must
not claim that you wrote the original software. If you use this
software in a product, an acknowledgment in the product
documentation would be appreciated but is not required.
2. Altered source versions must be plainly marked as such, and must
not be misrepresented as being the original software.
3. This notice may not be removed or altered from any source
distribution.
Marijn Haverbeke
marijnh at gmail

View File

@ -0,0 +1,32 @@
Copyright (c) 2009, Franciszek Wawrzak
All rights reserved.
This software is provided for use in connection with the
CodeMirror suite of modules and utilities, hosted and maintained
at http://marijn.haverbeke.nl/codemirror/.
Redistribution and use of this software in source and binary forms,
with or without modification, are permitted provided that the
following conditions are met:
* Redistributions of source code must retain the above
copyright notice, this list of conditions and the
following disclaimer.
* Redistributions in binary form must reproduce the above
copyright notice, this list of conditions and the
following disclaimer in the documentation and/or other
materials provided with the distribution.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN
ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
POSSIBILITY OF SUCH DAMAGE.

View File

@ -0,0 +1,59 @@
.editbox {
margin: .4em;
padding: 0;
font-family: monospace;
font-size: 10pt;
color: black;
}
pre.code, .editbox {
color: #666666;
}
.editbox p {
margin: 0;
}
span.lua-comment {
color: #BB9977;
}
span.lua-keyword {
font-weight: bold;
color: blue;
}
span.lua-string {
color: #AA2222;
}
span.lua-stdfunc {
font-weight: bold;
color: #077;
}
span.lua-customfunc {
font-weight: bold;
color: #0AA;
}
span.lua-identifier {
color: black;
}
span.lua-number {
color: #3A3;
}
span.lua-token {
color: #151;
}
span.lua-error {
color: #FFF;
background-color: #F00;
}

View File

@ -0,0 +1,68 @@
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<script src="../../js/codemirror.js" type="text/javascript"></script>
<title>CodeMirror: Lua demonstration</title>
</head>
<body style="padding: 20px;">
<p>This page demonstrates <a href="../../index.html">CodeMirror</a>'s
Lua parser. Written by <a href="http://francio.pl/">Franciszek
Wawrzak</a>, released under a BSD-style <a
href="LICENSE">license</a>.</p>
<div style="border: 1px solid black; padding: 0px;">
<textarea id="code" cols="120" rows="30">
--[[
example useless code to show lua syntax highlighting
this is multiline comment
]]
function blahblahblah(x)
local table = {
"asd" = 123,
"x" = 0.34,
}
if x ~= 3 then
print( x )
elseif x == "string"
my_custom_function( 0x34 )
else
unknown_function( "some string" )
end
--single line comment
end
function blablabla3()
for k,v in ipairs( table ) do
--abcde..
y=[=[
x=[[
x is a multi line string
]]
but its definition is iside a highest level string!
]=]
print(" \"\" ")
--this marks a parser error:
s = [== asdasdasd]]
s = math.sin( x )
end
end
</textarea>
</div>
<script type="text/javascript">
var editor = CodeMirror.fromTextArea('code', {
height: "350px",
parserfile: "../contrib/lua/js/parselua.js",
stylesheet: "css/luacolors.css",
path: "../../js/"
});
</script>
</body>
</html>

View File

@ -0,0 +1,253 @@
/*
Simple parser for LUA
Written for Lua 5.1, based on parsecss and other parsers.
features: highlights keywords, strings, comments (no leveling supported! ("[==[")),tokens, basic indenting
to make this parser highlight your special functions pass table with this functions names to parserConfig argument of creator,
parserConfig: ["myfunction1","myfunction2"],
*/
function findFirstRegexp(words) {
return new RegExp("^(?:" + words.join("|") + ")", "i");
}
function matchRegexp(words) {
return new RegExp("^(?:" + words.join("|") + ")$", "i");
}
var luaCustomFunctions= matchRegexp([]);
function configureLUA(parserConfig){
if(parserConfig)
luaCustomFunctions= matchRegexp(parserConfig);
}
//long list of standard functions from lua manual
var luaStdFunctions = matchRegexp([
"_G","_VERSION","assert","collectgarbage","dofile","error","getfenv","getmetatable","ipairs","load","loadfile","loadstring","module","next","pairs","pcall","print","rawequal","rawget","rawset","require","select","setfenv","setmetatable","tonumber","tostring","type","unpack","xpcall",
"coroutine.create","coroutine.resume","coroutine.running","coroutine.status","coroutine.wrap","coroutine.yield",
"debug.debug","debug.getfenv","debug.gethook","debug.getinfo","debug.getlocal","debug.getmetatable","debug.getregistry","debug.getupvalue","debug.setfenv","debug.sethook","debug.setlocal","debug.setmetatable","debug.setupvalue","debug.traceback",
"close","flush","lines","read","seek","setvbuf","write",
"io.close","io.flush","io.input","io.lines","io.open","io.output","io.popen","io.read","io.stderr","io.stdin","io.stdout","io.tmpfile","io.type","io.write",
"math.abs","math.acos","math.asin","math.atan","math.atan2","math.ceil","math.cos","math.cosh","math.deg","math.exp","math.floor","math.fmod","math.frexp","math.huge","math.ldexp","math.log","math.log10","math.max","math.min","math.modf","math.pi","math.pow","math.rad","math.random","math.randomseed","math.sin","math.sinh","math.sqrt","math.tan","math.tanh",
"os.clock","os.date","os.difftime","os.execute","os.exit","os.getenv","os.remove","os.rename","os.setlocale","os.time","os.tmpname",
"package.cpath","package.loaded","package.loaders","package.loadlib","package.path","package.preload","package.seeall",
"string.byte","string.char","string.dump","string.find","string.format","string.gmatch","string.gsub","string.len","string.lower","string.match","string.rep","string.reverse","string.sub","string.upper",
"table.concat","table.insert","table.maxn","table.remove","table.sort"
]);
var luaKeywords = matchRegexp(["and","break","elseif","false","nil","not","or","return",
"true","function", "end", "if", "then", "else", "do",
"while", "repeat", "until", "for", "in", "local" ]);
var luaIndentKeys = matchRegexp(["function", "if","repeat","for","while", "[\(]", "{"]);
var luaUnindentKeys = matchRegexp(["end", "until", "[\)]", "}"]);
var luaUnindentKeys2 = findFirstRegexp(["end", "until", "[\)]", "}"]);
var luaMiddleKeys = findFirstRegexp(["else","elseif"]);
var LUAParser = Editor.Parser = (function() {
var tokenizeLUA = (function() {
function normal(source, setState) {
var ch = source.next();
if (ch == "-" && source.equals("-")) {
source.next();
setState(inSLComment);
return null;
}
else if (ch == "\"" || ch == "'") {
setState(inString(ch));
return null;
}
if (ch == "[" && (source.equals("[") || source.equals("="))) {
var level = 0;
while(source.equals("=")){
level ++;
source.next();
}
if(! source.equals("[") )
return "lua-error";
setState(inMLSomething(level,"lua-string"));
return null;
}
else if (ch == "=") {
if (source.equals("="))
source.next();
return "lua-token";
}
else if (ch == ".") {
if (source.equals("."))
source.next();
if (source.equals("."))
source.next();
return "lua-token";
}
else if (ch == "+" || ch == "-" || ch == "*" || ch == "/" || ch == "%" || ch == "^" || ch == "#" ) {
return "lua-token";
}
else if (ch == ">" || ch == "<" || ch == "(" || ch == ")" || ch == "{" || ch == "}" || ch == "[" ) {
return "lua-token";
}
else if (ch == "]" || ch == ";" || ch == ":" || ch == ",") {
return "lua-token";
}
else if (source.equals("=") && (ch == "~" || ch == "<" || ch == ">")) {
source.next();
return "lua-token";
}
else if (/\d/.test(ch)) {
source.nextWhileMatches(/[\w.%]/);
return "lua-number";
}
else {
source.nextWhileMatches(/[\w\\\-_.]/);
return "lua-identifier";
}
}
function inSLComment(source, setState) {
var start = true;
var count=0;
while (!source.endOfLine()) {
var ch = source.next();
var level = 0;
if ((ch =="[") && start)
while(source.equals("=")){
source.next();
level++;
}
if (source.equals("[")){
setState(inMLSomething(level,"lua-comment"));
return null;
}
start = false;
}
setState(normal);
return "lua-comment";
}
function inMLSomething(level,what) {
//wat sholud be "lua-string" or "lua-comment", level is the number of "=" in opening mark.
return function(source, setState){
var dashes = 0;
while (!source.endOfLine()) {
var ch = source.next();
if (dashes == level+1 && ch == "]" ) {
setState(normal);
break;
}
if (dashes == 0)
dashes = (ch == "]") ? 1:0;
else
dashes = (ch == "=") ? dashes + 1 : 0;
}
return what;
}
}
function inString(quote) {
return function(source, setState) {
var escaped = false;
while (!source.endOfLine()) {
var ch = source.next();
if (ch == quote && !escaped)
break;
escaped = !escaped && ch == "\\";
}
if (!escaped)
setState(normal);
return "lua-string";
};
}
return function(source, startState) {
return tokenizer(source, startState || normal);
};
})();
function indentLUA(indentDepth, base) {
return function(nextChars) {
var closing = (luaUnindentKeys2.test(nextChars) || luaMiddleKeys.test(nextChars));
return base + ( indentUnit * (indentDepth - (closing?1:0)) );
};
}
function parseLUA(source,basecolumn) {
basecolumn = basecolumn || 0;
var tokens = tokenizeLUA(source);
var indentDepth = 0;
var iter = {
next: function() {
var token = tokens.next(), style = token.style, content = token.content;
if (style == "lua-identifier" && luaKeywords.test(content)){
token.style = "lua-keyword";
}
if (style == "lua-identifier" && luaStdFunctions.test(content)){
token.style = "lua-stdfunc";
}
if (style == "lua-identifier" && luaCustomFunctions.test(content)){
token.style = "lua-customfunc";
}
if (luaIndentKeys.test(content))
indentDepth++;
else if (luaUnindentKeys.test(content))
indentDepth--;
if (content == "\n")
token.indentation = indentLUA( indentDepth, basecolumn);
return token;
},
copy: function() {
var _tokenState = tokens.state, _indentDepth = indentDepth;
return function(source) {
tokens = tokenizeLUA(source, _tokenState);
indentDepth = _indentDepth;
return iter;
};
}
};
return iter;
}
return {make: parseLUA, configure:configureLUA, electricChars: "delf})"}; //en[d] els[e] unti[l] elsei[f] // this should be taken from Keys keywords
})();

View File

@ -0,0 +1,37 @@
Copyright (c) 2008-2009, Yahoo! Inc.
All rights reserved.
This software is provided for use in connection with the
CodeMirror suite of modules and utilities, hosted and maintained
at http://marijn.haverbeke.nl/codemirror/.
Redistribution and use of this software in source and binary forms,
with or without modification, are permitted provided that the
following conditions are met:
* Redistributions of source code must retain the above
copyright notice, this list of conditions and the
following disclaimer.
* Redistributions in binary form must reproduce the above
copyright notice, this list of conditions and the
following disclaimer in the documentation and/or other
materials provided with the distribution.
* Neither the name of Yahoo! Inc. nor the names of its
contributors may be used to endorse or promote products
derived from this software without specific prior
written permission of Yahoo! Inc.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN
ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
POSSIBILITY OF SUCH DAMAGE.

View File

@ -0,0 +1,110 @@
/*
Copyright (c) 2008-2009 Yahoo! Inc. All rights reserved.
The copyrights embodied in the content of this file are licensed by
Yahoo! Inc. under the BSD (revised) open source license
@author Dan Vlad Dascalescu <dandv@yahoo-inc.com>
*/
.editbox {
margin: .4em;
padding: 0;
font-family: monospace;
font-size: 10pt;
}
/*We should define specific styles for every element of the syntax.
the setting below will cause some annoying color to show through if we missed
defining a style for a token. This is also the "color" of the whitespace and
of the cursor.
*/
pre.code, .editbox {
color: red;
}
.editbox p {
margin: 0;
}
span.php-punctuation {
color: blue;
}
span.php-keyword {
color: #770088;
font-weight: bold;
}
span.php-operator {
color: blue;
}
/* __FILE__ etc.; http://php.net/manual/en/reserved.php */
span.php-compile-time-constant {
color: #776088;
font-weight: bold;
}
/* output of get_defined_constants(). Differs from http://php.net/manual/en/reserved.constants.php */
span.php-predefined-constant {
color: darkgreen;
font-weight: bold;
}
/* PHP reserved "language constructs"... echo() etc.; http://php.net/manual/en/reserved.php */
span.php-reserved-language-construct {
color: green;
font-weight: bold;
}
/* PHP built-in functions: glob(), chr() etc.; output of get_defined_functions()["internal"] */
span.php-predefined-function {
color: green;
}
/* PHP predefined classes: PDO, Exception etc.; output of get_declared_classes() and different from http://php.net/manual/en/reserved.classes.php */
span.php-predefined-class {
color: green;
}
span.php-atom {
color: #228811;
}
/* class, interface, namespace or function names, but not $variables */
span.php-t_string {
color: black;
}
span.php-variable {
color: black;
font-weight: bold;
}
span.js-localvariable {
color: #004499;
}
span.php-comment {
color: #AA7700;
font-stretch: condensed;
/* font-style: italic; This causes line height to slightly change, getting line numbers out of sync */
}
span.php-string-single-quoted {
color: #AA2222;
}
/* double quoted strings allow interpolation */
span.php-string-double-quoted {
color: #AA2222;
font-weight: bold;
}
span.syntax-error {
background-color: red;
}
span.deprecated {
font-size: smaller;
}

View File

@ -0,0 +1,292 @@
<!--
Copyright (c) 2008-2009 Yahoo! Inc. All rights reserved.
The copyrights embodied in the content of this file are licensed by
Yahoo! Inc. under the BSD (revised) open source license
@author Dan Vlad Dascalescu <dandv@yahoo-inc.com>
-->
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<script src="../../js/codemirror.js" type="text/javascript"></script>
<title>CodeMirror: PHP+HTML+JavaScript+CSS mixed-mode demonstration</title>
<link rel="stylesheet" type="text/css" href="../../css/docs.css"/>
</head>
<body style="padding: 20px;">
<p>This is a complex demonstration of the <b>PHP+HTML+JavaScript+CSS mixed-mode
syntax highlight</b> capabilities of <a href="../../index.html">CodeMirror</a>.
&lt;?php ... ?> tags use the PHP parser, &lt;script> tags use the JavaScript
parser, and &lt;style> tags use the CSS parser. The rest of the content is
parsed using the XML parser in HTML mode.</p>
<p>Features of the PHP parser:
<ul>
<li>special "deprecated" style for PHP4 keywords like 'var'
<li>support for PHP 5.3 keywords: 'namespace', 'use'
<li>911 predefined constants, 1301 predefined functions, 105 predeclared classes
from a typical PHP installation in a LAMP environment
<li>new feature: syntax error flagging, thus enabling strict parsing of:
<ol>
<li>function definitions with explicitly or implicitly typed arguments and default values
<li>modifiers (public, static etc.) applied to method and member definitions
<li>foreach(array_expression as $key [=> $value]) loops
</ol>
<li>differentiation between single-quoted strings and double-quoted interpolating strings
</ul>
</p>
<div style="border: 1px solid black; padding: 3px; background-color: #F8F8F8">
<textarea id="code" cols="120" rows="30">
The "root" parser is XML in HTML mode.
Next, we can switch into PHP mode, for example. This is
<?php echo 'text output by';
?>
PHP. </b>
On the line above, we just had an XML syntax error due to the </b> tag not being opened.
<?xml version='1.0' encoding='UTF-8' standalone='yes'?> HTML text will follow
<html>
<head>
<title>Similarly, the 'script' tag will switch to the JavaScript parser:</title>
<script type="text/javascript">
// Press enter inside the object and your new line will be suitably
// indented.
var keyBindings = {
enter: "newline-and-indent",
tab: "reindent-selection",
ctrl_enter: "reparse-buffer",
ctrl_z: "undo",
ctrl_y: "redo",
ctrl_backspace: "undo-for-safari-which-stupidly-enough-blocks-ctrl-z"
};
// Press tab on the next line and the wrong indentation will be fixed.
var regex = /foo|bar/i;
function example(x) {
// Local variables get a different colour than global ones.
var y = 44.4;
return x + y - z;
}
</script>
<style>
/* Some example CSS */
@import url("something.css");
body {
margin: 0;
padding: 3em 6em;
font-family: tahoma, arial, sans-serif;
color: #000;
}
#navigation a {
font-weight: bold;
text-decoration: none !important;
}
h1 {
font-size: 2.5em;
}
h1:before, h2:before {
content: "::";
}
code {
font-family: courier, monospace;
font-size: 80%;
color: #418A8A;
}
</style>
</head>
<body>
The PHP code below contains some deliberate errors. Play with the editor by fixing them
and observing how the highlight changes.
<?php
namespace A;
namespace A::B::C;
namespace A::::B;
namespace A::B::C::;
namespace A::B::C::D x;
self::range($row['lft'], $row['rgt'])); // error: extra ')'
$a = (b() + 4) 5 foo; // error: missing operators
self::$var;
$parent = self::range($max + 1, $max + 1);
$row[attributes][$attribute_name] = $attribute_value;
$row[attributes()][$attribute_name] = $attribute_value;
$row[attributes(5)][$attribute_name] = $attribute_value;
$row[$attributes()][$attribute_name] = $attribute_value;
abstract class 5 extends foo implements echo {
private function domainObjectBuilder() {
return $this->use_domain_object_builder
? $this->domain()->objectBuilder()
: null;
}
const $myconst = 'some string';
$array[myconst] = 4;
// this is a single-line C++-style comment
# this is a single-line shell-style comment
private var $a = __FILE__;
protected static $b = timezone_transitions_get('some parameter here');
global $g = isset("string");
static $s = hash_update_file; // warning: predefined function non-call
function mike ($var) $foo;
mike(A::func(param));
func($b $c); // error: function parameters must be comma-separated
foo bar; // error: no operator
$baz $quux; // error: no operator
public abstract function loadPageXML(util_FilePath $filename, $merge=0+$foo, $x, $y=3) {
$newrow[$key] = $val;
$newresult[] = $row;
$state = $row['c'] == 1;
$attribute_values[$attribute_name] = null;
$row['attributes'][$attribute_name] = $attribute_value;
$result[$row['element']][$row['attribute']] = $row['value'];
$sql = "multiline string
line2 is special - it'll interpolate variables like $state and method calls
{$this->cache->add($key, 5)} and maybe \"more\"
line5";
$sql = 'multiline string
single quoting means no \'interpolation\' like "$start" or method call
{$this->cache->add($key, 5)} will happen
line5';
$bitpattern = 1 << 2;
$bitpattern <<= 3;
$incorrect = <<< 5 EOSTRING // FIXME: CodeMirror update bug: add a letter before 5 and notice that syntax is not updated until EOF, even with continuousScanning: 500
error: the identifier must conform to the identifier rules
EOSTRING;
$sql = <<< EOSQL
SELECT attribute, element, value
FROM attribute_values
WHERE dimension = ?
EOSQL;
$this->lr_cache->add($key, self::range($row['lft'], $row['rgt']));
$composite_string = <<<EOSTRING
some lines here
EOSTRING
. 'something extra';
$page_lft = ($domain->name() == 'page') ? $start + 1 : $page_start + 1;
echo "This is class foo";
echo "a = ".$this ->a[2+3*$array["foo"]]."";
echo "b = {$this->b}"; // FIXME: highlight interpolation in strings
}
final function makecoffee error($types = array("cappuccino"), $coffeeMaker = NULL) {
$out_of_way_amount = $max - $child->left() + 1;
$absolute_pos = $child->left() - $move->width();
$varfunc(1, 'x');
$varfunc(1, 'x') + foo() - 5;
$funcarray[$i]('param1', $param2);
$lr[$domain_name] = $this->get_left_and_right($domain,
$dimension_name,
$element_name);
$domain_list = is_null($domain) ?
r3_Domain::names() :
array($domain->name());
foreach (r3_Domain::names() as $domain_name) {
$placeholders = 'distance LIKE '
. implode(array_fill(1, $num_distances, '?'),
' OR distance LIKE ');
}
return $this->target*$this->trans+myfunc(__METHOD__);
/*
echo 'This is a test'; /* This comment will cause a problem */
*/
}
switch( $type ) {
case "r3core_AddTemplateToTargetEvent":
$this->notifyAddTemplateToTarget( $genevent );
break;
case "r3core_GenerateTargetEvent" $this:
for($i=0; $i<=this->method(); $i++) {
echo 'Syntax "highlighting"';
}
try {
foreach($array xor $loader->parse_fn($filename) as $key => value) {
namespace r3;
}
} catch( Exception $e ) {
/** restore the backup
*/
$this->loadAll($tmp, $event, true);
// `php -l` doesn't complain at all at this (it assumes string constants):
this + makes * no - sense;
}
break;
default moo:
throw new r3_util_Exception( get_class( $genevent ) . " does not map" );
}
};
?>
<r3:cphp>
php("works", $here, 2);
</r3:cphp>
<r4:cphp>
class foo {
// a comment
var $a;
var $b;
};
</r4:cphp>
<h1>This is an <?php # echo 'simple';?> example.</h1>
<p>The header above will say 'This is an example'.</p>
<h1>This is an <?php // echo 'simple';?> example.</h1>
<?php echo; ?>
<body>
<?php echo "<html>
<head>
<script>
var foo = 'bar';
</script>
<style>
span.test {font-family: arial, 'lucida console', sans-serif}
</style>
</head>
<body>
<!-- comment -->
</body>
</html>"; ?>
</body>
</html>
</textarea>
</div>
<script type="text/javascript">
var editor = CodeMirror.fromTextArea('code', {
height: "350px",
parserfile: ["parsexml.js", "parsecss.js", "tokenizejavascript.js", "parsejavascript.js",
"../contrib/php/js/tokenizephp.js", "../contrib/php/js/parsephp.js",
"../contrib/php/js/parsephphtmlmixed.js"],
stylesheet: ["../../css/xmlcolors.css", "../../css/jscolors.css", "../../css/csscolors.css", "css/phpcolors.css"],
path: "../../js/",
continuousScanning: 500
});
</script>
</body>
</html>

View File

@ -0,0 +1,371 @@
/*
Copyright (c) 2008-2009 Yahoo! Inc. All rights reserved.
The copyrights embodied in the content of this file are licensed by
Yahoo! Inc. under the BSD (revised) open source license
@author Dan Vlad Dascalescu <dandv@yahoo-inc.com>
Parse function for PHP. Makes use of the tokenizer from tokenizephp.js.
Based on parsejavascript.js by Marijn Haverbeke.
Features:
+ special "deprecated" style for PHP4 keywords like 'var'
+ support for PHP 5.3 keywords: 'namespace', 'use'
+ 911 predefined constants, 1301 predefined functions, 105 predeclared classes
from a typical PHP installation in a LAMP environment
+ new feature: syntax error flagging, thus enabling strict parsing of:
+ function definitions with explicitly or implicitly typed arguments and default values
+ modifiers (public, static etc.) applied to method and member definitions
+ foreach(array_expression as $key [=> $value]) loops
+ differentiation between single-quoted strings and double-quoted interpolating strings
*/
// add the Array.indexOf method for JS engines that don't support it (e.g. IE)
// code from https://developer.mozilla.org/En/Core_JavaScript_1.5_Reference/Global_Objects/Array/IndexOf
if (!Array.prototype.indexOf)
{
Array.prototype.indexOf = function(elt /*, from*/)
{
var len = this.length;
var from = Number(arguments[1]) || 0;
from = (from < 0)
? Math.ceil(from)
: Math.floor(from);
if (from < 0)
from += len;
for (; from < len; from++)
{
if (from in this &&
this[from] === elt)
return from;
}
return -1;
};
};
var PHPParser = Editor.Parser = (function() {
// Token types that can be considered to be atoms, part of operator expressions
var atomicTypes = {
"atom": true, "number": true, "variable": true, "string": true
};
// Constructor for the lexical context objects.
function PHPLexical(indented, column, type, align, prev, info) {
// indentation at start of this line
this.indented = indented;
// column at which this scope was opened
this.column = column;
// type of scope ('stat' (statement), 'form' (special form), '[', '{', or '(')
this.type = type;
// '[', '{', or '(' blocks that have any text after their opening
// character are said to be 'aligned' -- any lines below are
// indented all the way to the opening character.
if (align != null)
this.align = align;
// Parent scope, if any.
this.prev = prev;
this.info = info;
};
// PHP indentation rules
function indentPHP(lexical) {
return function(firstChars) {
var firstChar = firstChars && firstChars.charAt(0), type = lexical.type;
var closing = firstChar == type;
if (type == "form" && firstChar == "{")
return lexical.indented;
else if (type == "stat" || type == "form")
return lexical.indented + indentUnit;
else if (lexical.info == "switch" && !closing)
return lexical.indented + (/^(?:case|default)\b/.test(firstChars) ? indentUnit : 2 * indentUnit);
else if (lexical.align)
return lexical.column - (closing ? 1 : 0);
else
return lexical.indented + (closing ? 0 : indentUnit);
};
};
// The parser-iterator-producing function itself.
function parsePHP(input, basecolumn) {
// Wrap the input in a token stream
var tokens = tokenizePHP(input);
// The parser state. cc is a stack of actions that have to be
// performed to finish the current statement. For example we might
// know that we still need to find a closing parenthesis and a
// semicolon. Actions at the end of the stack go first. It is
// initialized with an infinitely looping action that consumes
// whole statements.
var cc = [statements];
// The lexical scope, used mostly for indentation.
var lexical = new PHPLexical((basecolumn || 0) - indentUnit, 0, "block", false);
// Current column, and the indentation at the start of the current
// line. Used to create lexical scope objects.
var column = 0;
var indented = 0;
// Variables which are used by the mark, cont, and pass functions
// below to communicate with the driver loop in the 'next' function.
var consume, marked;
// The iterator object.
var parser = {next: next, copy: copy};
// parsing is accomplished by calling next() repeatedly
function next(){
// Start by performing any 'lexical' actions (adjusting the
// lexical variable), or the operations below will be working
// with the wrong lexical state.
while(cc[cc.length - 1].lex)
cc.pop()();
// Fetch the next token.
var token = tokens.next();
// Adjust column and indented.
if (token.type == "whitespace" && column == 0)
indented = token.value.length;
column += token.value.length;
if (token.content == "\n"){
indented = column = 0;
// If the lexical scope's align property is still undefined at
// the end of the line, it is an un-aligned scope.
if (!("align" in lexical))
lexical.align = false;
// Newline tokens get an indentation function associated with
// them.
token.indentation = indentPHP(lexical);
}
// No more processing for meaningless tokens.
if (token.type == "whitespace" || token.type == "comment"
|| token.type == "string_not_terminated" )
return token;
// When a meaningful token is found and the lexical scope's
// align is undefined, it is an aligned scope.
if (!("align" in lexical))
lexical.align = true;
// Execute actions until one 'consumes' the token and we can
// return it. 'marked' is used to change the style of the current token.
while(true) {
consume = marked = false;
// Take and execute the topmost action.
var action = cc.pop();
action(token);
if (consume){
if (marked)
token.style = marked;
// Here we differentiate between local and global variables.
return token;
}
}
return 1; // Firebug workaround for http://code.google.com/p/fbug/issues/detail?id=1239#c1
}
// This makes a copy of the parser state. It stores all the
// stateful variables in a closure, and returns a function that
// will restore them when called with a new input stream. Note
// that the cc array has to be copied, because it is contantly
// being modified. Lexical objects are not mutated, so they can
// be shared between runs of the parser.
function copy(){
var _lexical = lexical, _cc = cc.concat([]), _tokenState = tokens.state;
return function copyParser(input){
lexical = _lexical;
cc = _cc.concat([]); // copies the array
column = indented = 0;
tokens = tokenizePHP(input, _tokenState);
return parser;
};
}
// Helper function for pushing a number of actions onto the cc
// stack in reverse order.
function push(fs){
for (var i = fs.length - 1; i >= 0; i--)
cc.push(fs[i]);
}
// cont and pass are used by the action functions to add other
// actions to the stack. cont will cause the current token to be
// consumed, pass will leave it for the next action.
function cont(){
push(arguments);
consume = true;
}
function pass(){
push(arguments);
consume = false;
}
// Used to change the style of the current token.
function mark(style){
marked = style;
}
// Add a lyer of style to the current token, for example syntax-error
function mark_add(style){
marked = marked + ' ' + style;
}
// Push a new lexical context of the given type.
function pushlex(type, info) {
var result = function pushlexing() {
lexical = new PHPLexical(indented, column, type, null, lexical, info)
};
result.lex = true;
return result;
}
// Pop off the current lexical context.
function poplex(){
lexical = lexical.prev;
}
poplex.lex = true;
// The 'lex' flag on these actions is used by the 'next' function
// to know they can (and have to) be ran before moving on to the
// next token.
// Creates an action that discards tokens until it finds one of
// the given type. This will ignore (and recover from) syntax errors.
function expect(wanted){
return function expecting(token){
if (token.type == wanted) cont(); // consume the token
else {
cont(arguments.callee); // continue expecting() - call itself
}
};
}
// Require a specific token type, or one of the tokens passed in the 'wanted' array
// Used to detect blatant syntax errors. 'execute' is used to pass extra code
// to be executed if the token is matched. For example, a '(' match could
// 'execute' a cont( compasep(funcarg), require(")") )
function require(wanted, execute){
return function requiring(token){
var ok;
var type = token.type;
if (typeof(wanted) == "string")
ok = (type == wanted) -1;
else
ok = wanted.indexOf(type);
if (ok >= 0) {
if (execute && typeof(execute[ok]) == "function")
execute[ok](token);
cont(); // just consume the token
}
else {
if (!marked) mark(token.style);
mark_add("syntax-error");
cont(arguments.callee);
}
};
}
// Looks for a statement, and then calls itself.
function statements(token){
return pass(statement, statements);
}
// Dispatches various types of statements based on the type of the current token.
function statement(token){
var type = token.type;
if (type == "keyword a") cont(pushlex("form"), expression, statement, poplex);
else if (type == "keyword b") cont(pushlex("form"), statement, poplex);
else if (type == "{") cont(pushlex("}"), block, poplex);
else if (type == "function") funcdef();
// technically, "class implode {...}" is correct, but we'll flag that as an error because it overrides a predefined function
else if (type == "class") cont(require("t_string"), expect("{"), pushlex("}"), block, poplex);
else if (type == "foreach") cont(pushlex("form"), require("("), pushlex(")"), expression, require("as"), require("variable"), /* => $value */ expect(")"), poplex, statement, poplex);
else if (type == "for") cont(pushlex("form"), require("("), pushlex(")"), expression, require(";"), expression, require(";"), expression, require(")"), poplex, statement, poplex);
// public final function foo(), protected static $bar;
else if (type == "modifier") cont(require(["modifier", "variable", "function"], [null, null, funcdef]));
else if (type == "switch") cont(pushlex("form"), require("("), expression, require(")"), pushlex("}", "switch"), require([":", "{"]), block, poplex, poplex);
else if (type == "case") cont(expression, require(":"));
else if (type == "default") cont(require(":"));
else if (type == "catch") cont(pushlex("form"), require("("), require("t_string"), require("variable"), require(")"), statement, poplex);
else if (type == "const") cont(require("t_string")); // 'const static x=5' is a syntax error
// technically, "namespace implode {...}" is correct, but we'll flag that as an error because it overrides a predefined function
else if (type == "namespace") cont(namespacedef, require(";"));
// $variables may be followed by operators, () for variable function calls, or [] subscripts
else pass(pushlex("stat"), expression, require(";"), poplex);
}
// Dispatch expression types.
function expression(token){
var type = token.type;
if (atomicTypes.hasOwnProperty(type)) cont(maybeoperator);
else if (type == "<<<") cont(require("string"), maybeoperator); // heredoc/nowdoc
else if (type == "t_string") cont(maybe_double_colon, maybeoperator);
else if (type == "keyword c") cont(expression);
// function call or parenthesized expression: $a = ($b + 1) * 2;
else if (type == "(") cont(pushlex(")"), commasep(expression), require(")"), poplex, maybeoperator);
else if (type == "operator") cont(expression);
}
// Called for places where operators, function calls, or subscripts are
// valid. Will skip on to the next action if none is found.
function maybeoperator(token){
var type = token.type;
if (type == "operator") {
if (token.content == "?") cont(expression, require(":"), expression); // ternary operator
else cont(expression);
}
else if (type == "(") cont(pushlex(")"), expression, commasep(expression), require(")"), poplex, maybeoperator /* $varfunc() + 3 */);
else if (type == "[") cont(pushlex("]"), expression, require("]"), maybeoperator /* for multidimensional arrays, or $func[$i]() */, poplex);
}
// A regular use of the double colon to specify a class, as in self::func() or myclass::$var;
// Differs from `namespace` or `use` in that only one class can be the parent; chains (A::B::$var) are a syntax error.
function maybe_double_colon(token) {
if (token.type == "t_double_colon")
// A::$var, A::func(), A::const
cont(require(["t_string", "variable"]), maybeoperator);
else {
// a t_string wasn't followed by ::, such as in a function call: foo()
pass(expression)
}
}
// the declaration or definition of a function
function funcdef() {
cont(require("t_string"), require("("), pushlex(")"), commasep(funcarg), require(")"), poplex, block);
}
// Parses a comma-separated list of the things that are recognized
// by the 'what' argument.
function commasep(what){
function proceed(token) {
if (token.type == ",") cont(what, proceed);
};
return function commaSeparated() {
pass(what, proceed);
};
}
// Look for statements until a closing brace is found.
function block(token) {
if (token.type == "}") cont();
else pass(statement, block);
}
function maybedefaultparameter(token){
if (token.content == "=") cont(expression);
}
// support for default arguments: http://us.php.net/manual/en/functions.arguments.php#functions.arguments.default
function funcarg(token){
// function foo(myclass $obj) {...}
if (token.type == "t_string") cont(require("variable"), maybedefaultparameter);
// function foo($string) {...}
else if (token.type == "variable") cont(maybedefaultparameter);
}
// A namespace definition or use
function maybe_double_colon_def(token) {
if (token.type == "t_double_colon")
cont(namespacedef);
}
function namespacedef(token) {
pass(require("t_string"), maybe_double_colon_def);
}
return parser;
}
return {make: parsePHP, electricChars: "{}:"};
})();

View File

@ -0,0 +1,90 @@
/*
Copyright (c) 2008-2009 Yahoo! Inc. All rights reserved.
The copyrights embodied in the content of this file are licensed by
Yahoo! Inc. under the BSD (revised) open source license
@author Dan Vlad Dascalescu <dandv@yahoo-inc.com>
Based on parsehtmlmixed.js by Marijn Haverbeke.
*/
var PHPHTMLMixedParser = Editor.Parser = (function() {
if (!(PHPParser && CSSParser && JSParser && XMLParser))
throw new Error("PHP, CSS, JS, and XML parsers must be loaded for PHP+HTML mixed mode to work.");
XMLParser.configure({useHTMLKludges: true});
function parseMixed(stream) {
var htmlParser = XMLParser.make(stream), localParser = null, inTag = false;
var iter = {next: top, copy: copy};
function top() {
var token = htmlParser.next();
if (token.content == "<")
inTag = true;
else if (token.style == "xml-tagname" && inTag === true)
inTag = token.content.toLowerCase();
else if (token.type == "xml-processing") {
// dispatch on PHP
if (token.content == "<?php")
iter.next = local(PHPParser, "?>");
}
// "xml-processing" tokens are ignored, because they should be handled by a specific local parser
else if (token.content == ">") {
if (inTag == "script")
iter.next = local(JSParser, "</script");
else if (inTag == "style")
iter.next = local(CSSParser, "</style");
inTag = false;
}
return token;
}
function local(parser, tag) {
var baseIndent = htmlParser.indentation();
localParser = parser.make(stream, baseIndent + indentUnit);
return function() {
if (stream.lookAhead(tag, false, false, true)) {
localParser = null;
iter.next = top;
return top(); // pass the ending tag to the enclosing parser
}
var token = localParser.next();
var lt = token.value.lastIndexOf("<"), sz = Math.min(token.value.length - lt, tag.length);
if (lt != -1 && token.value.slice(lt, lt + sz).toLowerCase() == tag.slice(0, sz) &&
stream.lookAhead(tag.slice(sz), false, false, true)) {
stream.push(token.value.slice(lt));
token.value = token.value.slice(0, lt);
}
if (token.indentation) {
var oldIndent = token.indentation;
token.indentation = function(chars) {
if (chars == "</")
return baseIndent;
else
return oldIndent(chars);
}
}
return token;
};
}
function copy() {
var _html = htmlParser.copy(), _local = localParser && localParser.copy(),
_next = iter.next, _inTag = inTag;
return function(_stream) {
stream = _stream;
htmlParser = _html(_stream);
localParser = _local && _local(_stream);
iter.next = _next;
inTag = _inTag;
return iter;
};
}
return iter;
}
return {make: parseMixed, electricChars: "{}/:"};
})();

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,32 @@
Copyright (c) 2009, Timothy Farrell
All rights reserved.
This software is provided for use in connection with the
CodeMirror suite of modules and utilities, hosted and maintained
at http://marijn.haverbeke.nl/codemirror/.
Redistribution and use of this software in source and binary forms,
with or without modification, are permitted provided that the
following conditions are met:
* Redistributions of source code must retain the above
copyright notice, this list of conditions and the
following disclaimer.
* Redistributions in binary form must reproduce the above
copyright notice, this list of conditions and the
following disclaimer in the documentation and/or other
materials provided with the distribution.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN
ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
POSSIBILITY OF SUCH DAMAGE.

View File

@ -0,0 +1,54 @@
.editbox {
padding: .4em;
margin: 0;
font-family: monospace;
font-size: 10pt;
line-height: 1.1em;
color: black;
}
pre.code, .editbox {
color: #666666;
}
.editbox p {
margin: 0;
}
span.py-delimiter, span.py-special {
color: #666666;
}
span.py-operator {
color: #666666;
}
span.py-error {
background-color: #660000;
color: #FFFFFF;
}
span.py-keyword {
color: #770088;
font-weight: bold;
}
span.py-literal {
color: #228811;
}
span.py-identifier, span.py-func {
color: black;
}
span.py-type, span.py-decorator {
color: #0000FF;
}
span.py-comment {
color: #AA7700;
}
span.py-string, span.py-bytes, span.py-raw, span.py-unicode {
color: #AA2222;
}

View File

@ -0,0 +1,141 @@
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<script src="../../js/codemirror.js" type="text/javascript"></script>
<title>CodeMirror: Python demonstration</title>
<style type="text/css">
.CodeMirror-line-numbers {
width: 2.2em;
color: #aaa;
background-color: #eee;
text-align: right;
padding: .4em;
margin: 0;
font-family: monospace;
font-size: 10pt;
line-height: 1.1em;
}
</style>
</head>
<body style="padding: 20px;">
<p>
This is a simple demonstration of the Python syntax highlighting module
for <a href="index.html">CodeMirror</a>.
</p>
<p>
Features of this parser include:
</p>
<ul>
<li>Token-based syntax highlighting - currently very little lexical analysis happens. Few lexical errors will be detected.</li>
<li>Use the normal indentation mode to enforce regular indentation, otherwise the "shift" indentation mode will give you more flexibility.</li>
<li>Parser Options:
<ul>
<li>pythonVersion (Integer) - 2 or 3 to indicate which version of Python to parse. Default = 2</li>
<li>strictErrors (Bool) - true to highlight errors that may not be Python errors but cause confusion for this parser. Default = true</li>
</ul>
</li>
</ul>
<p>Written by Timothy Farrell (<a href="LICENSE">license</a>). Special
thanks to Adam Brand and Marijn Haverbeke for their help in debugging
and providing for this parser.</p>
<div style="border: 1px solid black; padding: 0px;">
<textarea id="code" cols="100" rows="20" style="width:100%">
# Literals
1234
0.0e101
.123
0b01010011100
0o01234567
0x0987654321abcdef
# Error Literals
.0b000
0.0e
0e
# String Literals
'For\''
"God\""
"""so loved
the world"""
'''that he gave
his only begotten\' '''
'that whosoever believeth \
in him'
''
# Identifiers
__a__
a.b
a.b.c
# Error Identifiers
a.
# Operators
+ - * / % & | ^ ~ < >
== != <= >= <> << >> // **
and or not in is
# Delimiters
() [] {} , : ` = ; @ . # At-signs and periods require context
+= -= *= /= %= &= |= ^=
//= >>= <<= **=
# Keywords
as assert break class continue def del elif else except
finally for from global if import lambda pass raise
return try while with yield
# Python 2 Keywords (otherwise Identifiers)
exec print
# Python 3 Keywords (otherwise Identifiers)
nonlocal
# Types
bool classmethod complex dict enumerate float frozenset int list object
property reversed set slice staticmethod str super tuple type
# Python 2 Types (otherwise Identifiers)
basestring buffer file long unicode xrange
# Python 3 Types (otherwise Identifiers)
bytearray bytes filter map memoryview open range zip
# Example Strict Errors
def doesNothing():
pass # indentUnit is set to 4 but this line is indented 3
# Some Example code
import os
from package import ParentClass
@nonsenseDecorator
def doesNothing():
pass
class ExampleClass(ParentClass):
@staticmethod
def example(inputStr):
a = list(inputStr)
a.reverse()
return ''.join(a)
def __init__(self, mixin = 'Hello'):
self.mixin = mixin
</textarea>
</div>
<script type="text/javascript">
var editor = CodeMirror.fromTextArea('code', {
parserfile: ["../contrib/python/js/parsepython.js"],
stylesheet: "css/pythoncolors.css",
path: "../../js/",
lineNumbers: true,
textWrapping: false,
indentUnit: 4,
parserConfig: {'pythonVersion': 2, 'strictErrors': true}
});
</script>
</body>
</html>

View File

@ -0,0 +1,544 @@
var PythonParser = Editor.Parser = (function() {
function wordRegexp(words) {
return new RegExp("^(?:" + words.join("|") + ")$");
}
var DELIMITERCLASS = 'py-delimiter';
var LITERALCLASS = 'py-literal';
var ERRORCLASS = 'py-error';
var OPERATORCLASS = 'py-operator';
var IDENTIFIERCLASS = 'py-identifier';
var STRINGCLASS = 'py-string';
var BYTESCLASS = 'py-bytes';
var UNICODECLASS = 'py-unicode';
var RAWCLASS = 'py-raw';
var NORMALCONTEXT = 'normal';
var STRINGCONTEXT = 'string';
var singleOperators = '+-*/%&|^~<>';
var doubleOperators = wordRegexp(['==', '!=', '\\<=', '\\>=', '\\<\\>',
'\\<\\<', '\\>\\>', '\\/\\/', '\\*\\*']);
var singleDelimiters = '()[]{}@,:.`=;';
var doubleDelimiters = ['\\+=', '\\-=', '\\*=', '/=', '%=', '&=', '\\|=',
'\\^='];
var tripleDelimiters = wordRegexp(['//=','\\>\\>=','\\<\\<=','\\*\\*=']);
var singleStarters = singleOperators + singleDelimiters + '=!';
var doubleStarters = '=<>*/';
var identifierStarters = /[_A-Za-z]/;
var wordOperators = wordRegexp(['and', 'or', 'not', 'is', 'in']);
var commonkeywords = ['as', 'assert', 'break', 'class', 'continue',
'def', 'del', 'elif', 'else', 'except', 'finally',
'for', 'from', 'global', 'if', 'import',
'lambda', 'pass', 'raise', 'return',
'try', 'while', 'with', 'yield'];
var commontypes = ['bool', 'classmethod', 'complex', 'dict', 'enumerate',
'float', 'frozenset', 'int', 'list', 'object',
'property', 'reversed', 'set', 'slice', 'staticmethod',
'str', 'super', 'tuple', 'type'];
var py2 = {'types': ['basestring', 'buffer', 'file', 'long', 'unicode',
'xrange'],
'keywords': ['exec', 'print'],
'version': 2 };
var py3 = {'types': ['bytearray', 'bytes', 'filter', 'map', 'memoryview',
'open', 'range', 'zip'],
'keywords': ['nonlocal'],
'version': 3};
var py, keywords, types, stringStarters, stringTypes, config;
function configure(conf) {
if (!conf.hasOwnProperty('pythonVersion')) {
conf.pythonVersion = 2;
}
if (!conf.hasOwnProperty('strictErrors')) {
conf.strictErrors = true;
}
if (conf.pythonVersion != 2 && conf.pythonVersion != 3) {
alert('CodeMirror: Unknown Python Version "' +
conf.pythonVersion +
'", defaulting to Python 2.x.');
conf.pythonVersion = 2;
}
if (conf.pythonVersion == 3) {
py = py3;
stringStarters = /[\'\"rbRB]/;
stringTypes = /[rb]/;
doubleDelimiters.push('\\-\\>');
} else {
py = py2;
stringStarters = /['"RUru]/;
stringTypes = /[ru]/;
}
config = conf;
keywords = wordRegexp(commonkeywords.concat(py.keywords));
types = wordRegexp(commontypes.concat(py.types));
doubleDelimiters = wordRegexp(doubleDelimiters);
}
var tokenizePython = (function() {
function normal(source, setState) {
var stringDelim, threeStr, temp, type, word, possible = {};
var ch = source.next();
function filterPossible(token, styleIfPossible) {
if (!possible.style && !possible.content) {
return token;
} else if (typeof(token) == STRINGCONTEXT) {
token = {content: source.get(), style: token};
}
if (possible.style || styleIfPossible) {
token.style = styleIfPossible ? styleIfPossible : possible.style;
}
if (possible.content) {
token.content = possible.content + token.content;
}
possible = {};
return token;
}
// Handle comments
if (ch == '#') {
while (!source.endOfLine()) {
source.next();
}
return 'py-comment';
}
// Handle special chars
if (ch == '\\') {
if (source.peek() != '\n') {
var whitespace = true;
while (!source.endOfLine()) {
if(!(/\s/.test(source.next()))) {
whitespace = false;
}
}
if (!whitespace) {
return ERRORCLASS;
}
}
return 'py-special';
}
// Handle operators and delimiters
if (singleStarters.indexOf(ch) != -1) {
if (doubleStarters.indexOf(source.peek()) != -1) {
temp = ch + source.peek();
// It must be a double delimiter or operator or triple delimiter
if (doubleOperators.test(temp)) {
source.next();
if (tripleDelimiters.test(temp + source.peek())) {
source.next();
return DELIMITERCLASS;
} else {
return OPERATORCLASS;
}
} else if (doubleDelimiters.test(temp)) {
source.next();
return DELIMITERCLASS;
}
}
// It must be a single delimiter or operator
if (singleOperators.indexOf(ch) != -1) {
return OPERATORCLASS;
} else if (singleDelimiters.indexOf(ch) != -1) {
if (ch == '@' && /\w/.test(source.peek())) {
possible = {style:'py-decorator',
content: source.get()};
ch = source.next();
} else if (ch == '.' && /\d/.test(source.peek())) {
possible = {style:LITERALCLASS,
content: source.get()};
ch = source.next();
} else {
return DELIMITERCLASS;
}
} else {
return ERRORCLASS;
}
}
// Handle number literals
if (/\d/.test(ch)) {
if (ch === '0' && !source.endOfLine()) {
switch (source.peek()) {
case 'o':
case 'O':
source.next();
source.nextWhileMatches(/[0-7]/);
return filterPossible(LITERALCLASS, ERRORCLASS);
case 'x':
case 'X':
source.next();
source.nextWhileMatches(/[0-9A-Fa-f]/);
return filterPossible(LITERALCLASS, ERRORCLASS);
case 'b':
case 'B':
source.next();
source.nextWhileMatches(/[01]/);
return filterPossible(LITERALCLASS, ERRORCLASS);
}
}
source.nextWhileMatches(/\d/);
if (source.peek() == '.') {
source.next();
source.nextWhileMatches(/\d/);
}
// Grab an exponent
if (source.peek().toLowerCase() == 'e') {
source.next();
if (source.peek() == '+' || source.peek() == '-') {
source.next();
}
if (/\d/.test(source.peek())) {
source.nextWhileMatches(/\d/);
} else {
return filterPossible(ERRORCLASS);
}
}
// Grab a complex number
if (source.peek().toLowerCase() == 'j') {
source.next();
}
return filterPossible(LITERALCLASS);
}
// Handle strings
if (stringStarters.test(ch)) {
var peek = source.peek();
var stringType = STRINGCLASS;
if ((stringTypes.test(ch)) && (peek == '"' || peek == "'")) {
switch (ch.toLowerCase()) {
case 'b':
stringType = BYTESCLASS;
break;
case 'r':
stringType = RAWCLASS;
break;
case 'u':
stringType = UNICODECLASS;
break;
}
ch = source.next();
stringDelim = ch;
if (source.peek() != stringDelim) {
setState(inString(stringType, stringDelim));
return null;
} else {
source.next();
if (source.peek() == stringDelim) {
source.next();
threeStr = stringDelim + stringDelim + stringDelim;
setState(inString(stringType, threeStr));
return null;
} else {
return stringType;
}
}
} else if (ch == "'" || ch == '"') {
stringDelim = ch;
if (source.peek() != stringDelim) {
setState(inString(stringType, stringDelim));
return null;
} else {
source.next();
if (source.peek() == stringDelim) {
source.next();
threeStr = stringDelim + stringDelim + stringDelim;
setState(inString(stringType, threeStr));
return null;
} else {
return stringType;
}
}
}
}
// Handle Identifier
if (identifierStarters.test(ch)) {
source.nextWhileMatches(/[\w\d]/);
word = source.get();
if (wordOperators.test(word)) {
type = OPERATORCLASS;
} else if (keywords.test(word)) {
type = 'py-keyword';
} else if (types.test(word)) {
type = 'py-type';
} else {
type = IDENTIFIERCLASS;
while (source.peek() == '.') {
source.next();
if (identifierStarters.test(source.peek())) {
source.nextWhileMatches(/[\w\d]/);
} else {
type = ERRORCLASS;
break;
}
}
word = word + source.get();
}
return filterPossible({style: type, content: word});
}
// Register Dollar sign and Question mark as errors. Always!
if (/\$\?/.test(ch)) {
return filterPossible(ERRORCLASS);
}
return filterPossible(ERRORCLASS);
}
function inString(style, terminator) {
return function(source, setState) {
var matches = [];
var found = false;
while (!found && !source.endOfLine()) {
var ch = source.next(), newMatches = [];
// Skip escaped characters
if (ch == '\\') {
if (source.peek() == '\n') {
break;
}
ch = source.next();
ch = source.next();
}
if (ch == terminator.charAt(0)) {
matches.push(terminator);
}
for (var i = 0; i < matches.length; i++) {
var match = matches[i];
if (match.charAt(0) == ch) {
if (match.length == 1) {
setState(normal);
found = true;
break;
} else {
newMatches.push(match.slice(1));
}
}
}
matches = newMatches;
}
return style;
};
}
return function(source, startState) {
return tokenizer(source, startState || normal);
};
})();
function parsePython(source) {
if (!keywords) {
configure({});
}
var tokens = tokenizePython(source);
var lastToken = null;
var column = 0;
var context = {prev: null,
endOfScope: false,
startNewScope: false,
level: 0,
next: null,
type: NORMALCONTEXT
};
function pushContext(level, type) {
type = type ? type : NORMALCONTEXT;
context = {prev: context,
endOfScope: false,
startNewScope: false,
level: level,
next: null,
type: type
};
}
function popContext(remove) {
remove = remove ? remove : false;
if (context.prev) {
if (remove) {
context = context.prev;
context.next = null;
} else {
context.prev.next = context;
context = context.prev;
}
}
}
function indentPython(context) {
var temp;
return function(nextChars, currentLevel, direction) {
if (direction === null || direction === undefined) {
if (nextChars) {
while (context.next) {
context = context.next;
}
}
return context.level;
}
else if (direction === true) {
if (currentLevel == context.level) {
if (context.next) {
return context.next.level;
} else {
return context.level;
}
} else {
temp = context;
while (temp.prev && temp.prev.level > currentLevel) {
temp = temp.prev;
}
return temp.level;
}
} else if (direction === false) {
if (currentLevel > context.level) {
return context.level;
} else if (context.prev) {
temp = context;
while (temp.prev && temp.prev.level >= currentLevel) {
temp = temp.prev;
}
if (temp.prev) {
return temp.prev.level;
} else {
return temp.level;
}
}
}
return context.level;
};
}
var iter = {
next: function() {
var token = tokens.next();
var type = token.style;
var content = token.content;
if (lastToken) {
if (lastToken.content == 'def' && type == IDENTIFIERCLASS) {
token.style = 'py-func';
}
if (lastToken.content == '\n') {
var tempCtx = context;
// Check for a different scope
if (type == 'whitespace' && context.type == NORMALCONTEXT) {
if (token.value.length < context.level) {
while (token.value.length < context.level) {
popContext();
}
if (token.value.length != context.level) {
context = tempCtx;
if (config.strictErrors) {
token.style = ERRORCLASS;
}
} else {
context.next = null;
}
}
} else if (context.level !== 0 &&
context.type == NORMALCONTEXT) {
while (0 !== context.level) {
popContext();
}
if (context.level !== 0) {
context = tempCtx;
if (config.strictErrors) {
token.style = ERRORCLASS;
}
}
}
}
}
// Handle Scope Changes
switch(type) {
case STRINGCLASS:
case BYTESCLASS:
case RAWCLASS:
case UNICODECLASS:
if (context.type !== STRINGCONTEXT) {
pushContext(context.level + 1, STRINGCONTEXT);
}
break;
default:
if (context.type === STRINGCONTEXT) {
popContext(true);
}
break;
}
switch(content) {
case '.':
case '@':
// These delimiters don't appear by themselves
if (content !== token.value) {
token.style = ERRORCLASS;
}
break;
case ':':
// Colons only delimit scope inside a normal scope
if (context.type === NORMALCONTEXT) {
context.startNewScope = context.level+indentUnit;
}
break;
case '(':
case '[':
case '{':
// These start a sequence scope
pushContext(column + content.length, 'sequence');
break;
case ')':
case ']':
case '}':
// These end a sequence scope
popContext(true);
break;
case 'pass':
case 'return':
// These end a normal scope
if (context.type === NORMALCONTEXT) {
context.endOfScope = true;
}
break;
case '\n':
// Reset our column
column = 0;
// Make any scope changes
if (context.endOfScope) {
context.endOfScope = false;
popContext();
} else if (context.startNewScope !== false) {
var temp = context.startNewScope;
context.startNewScope = false;
pushContext(temp, NORMALCONTEXT);
}
// Newlines require an indentation function wrapped in a closure for proper context.
token.indentation = indentPython(context);
break;
}
// Keep track of current column for certain scopes.
if (content != '\n') {
column += token.value.length;
}
lastToken = token;
return token;
},
copy: function() {
var _context = context, _tokenState = tokens.state;
return function(source) {
tokens = tokenizePython(source, _tokenState);
context = _context;
return iter;
};
}
};
return iter;
}
return {make: parsePython,
electricChars: "",
configure: configure};
})();

View File

@ -0,0 +1,47 @@
.editbox {
margin: .4em;
padding: 0;
font-family: monospace;
font-size: 10pt;
color: black;
}
pre.code, .editbox {
color: #666666;
}
.editbox p {
margin: 0;
}
span.css-at {
color: #770088;
}
span.css-unit {
color: #228811;
}
span.css-value {
color: #770088;
}
span.css-identifier {
color: black;
}
span.css-important {
color: #0000FF;
}
span.css-colorcode {
color: #004499;
}
span.css-comment {
color: #AA7700;
}
span.css-string {
color: #AA2222;
}

View File

@ -0,0 +1,46 @@
body {
margin: 0;
padding: 3em 6em;
color: black;
max-width: 50em;
}
h1 {
font-size: 22pt;
}
.underline {
border-bottom: 3px solid #C44;
}
h2 {
font-size: 14pt;
}
p.rel {
padding-left: 2em;
text-indent: -2em;
}
div.border {
border: 1px solid black;
padding: 3px;
}
code {
font-family: courier, monospace;
font-size: 90%;
color: #144;
}
pre.code {
margin: 1.1em 12px;
border: 1px solid #CCCCCC;
color: black;
padding: .4em;
font-family: courier, monospace;
}
.warn {
color: #C00;
}

View File

@ -0,0 +1,55 @@
.editbox {
margin: .4em;
padding: 0;
font-family: monospace;
font-size: 10pt;
color: black;
}
pre.code, .editbox {
color: #666666;
}
.editbox p {
margin: 0;
}
span.js-punctuation {
color: #666666;
}
span.js-operator {
color: #666666;
}
span.js-keyword {
color: #770088;
}
span.js-atom {
color: #228811;
}
span.js-variable {
color: black;
}
span.js-variabledef {
color: #0000FF;
}
span.js-localvariable {
color: #004499;
}
span.js-property {
color: black;
}
span.js-comment {
color: #AA7700;
}
span.js-string {
color: #AA2222;
}

Binary file not shown.

After

Width:  |  Height:  |  Size: 14 KiB

View File

@ -0,0 +1,39 @@
.editbox {
margin: .4em;
padding: 0;
font-family: monospace;
font-size: 10pt;
color: black;
}
.editbox p {
margin: 0;
}
span.sp-keyword {
color: #708;
}
span.sp-prefixed {
color: #5d1;
}
span.sp-var {
color: #00c;
}
span.sp-comment {
color: #a70;
}
span.sp-literal {
color: #a22;
}
span.sp-uri {
color: #292;
}
span.sp-operator {
color: #088;
}

View File

@ -0,0 +1,70 @@
.editbox {
margin: .4em;
padding: 0;
font-family: monospace;
font-size: 10pt;
color: black;
}
pre.code, .editbox {
color: #666666;
}
.editbox p {
margin: 0;
}
span.svx-command {
color: red;
}
span.svx-begin, span.svx-end {
color: #990000;
}
span.svx-comment {
color: blue;
}
span.svx-measure {
color: green;
}
span.svx-station {
color: #009900;
}
span.svx-string {
color: blue;
}
span.svx-word {
color: black;
}
span.css-at {
color: #770088;
}
span.css-unit {
color: #228811;
}
span.css-value {
color: #770088;
}
span.css-identifier {
color: black;
}
span.css-important {
color: #0000FF;
}
span.css-colorcode {
color: #004499;
}
span.css-comment {
color: #AA7700;
}
span.css-string {
color: #AA2222;
}

View File

@ -0,0 +1,51 @@
.editbox {
margin: .4em;
padding: 0;
font-family: monospace;
font-size: 10pt;
color: black;
}
.editbox p {
margin: 0;
}
span.xml-tagname {
color: #A0B;
}
span.xml-attribute {
color: #281;
}
span.xml-punctuation {
color: black;
}
span.xml-attname {
color: #00F;
}
span.xml-comment {
color: #A70;
}
span.xml-cdata {
color: #48A;
}
span.xml-processing {
color: #999;
}
span.xml-entity {
color: #A22;
}
span.xml-error {
color: #F00;
}
span.xml-text {
color: black;
}

View File

@ -0,0 +1,60 @@
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<script src="js/codemirror.js" type="text/javascript"></script>
<title>CodeMirror: CSS demonstration</title>
<link rel="stylesheet" type="text/css" href="css/docs.css"/>
</head>
<body style="padding: 20px;">
<p>Demonstration of <a href="index.html">CodeMirror</a>'s CSS
highlighter.</p>
<div class="border">
<textarea id="code" cols="120" rows="30">
/* Some example CSS */
@import url("something.css");
body {
margin: 0;
padding: 3em 6em;
font-family: tahoma, arial, sans-serif;
color: #000;
}
#navigation a {
font-weight: bold;
text-decoration: none !important;
}
h1 {
font-size: 2.5em;
}
h2 {
font-size: 1.7em;
}
h1:before, h2:before {
content: "::";
}
code {
font-family: courier, monospace;
font-size: 80%;
color: #418A8A;
}
</textarea>
</div>
<script type="text/javascript">
var editor = CodeMirror.fromTextArea('code', {
height: "350px",
parserfile: "parsecss.js",
stylesheet: "css/csscolors.css",
path: "js/"
});
</script>
</body>
</html>

View File

@ -0,0 +1,82 @@
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<script src="js/highlight.js" type="text/javascript"></script>
<script src="js/stringstream.js" type="text/javascript"></script>
<script src="js/tokenize.js" type="text/javascript"></script>
<script src="js/tokenizejavascript.js" type="text/javascript"></script>
<script src="js/parsejavascript.js" type="text/javascript"></script>
<title>CodeMirror: String highlight demonstration</title>
<link rel="stylesheet" type="text/css" href="css/jscolors.css"/>
</head>
<body style="padding: 20px;">
<div style="border: 1px solid black; padding: .4em;">
<textarea id="code" cols="120" rows="20" style="border-width: 0">
// Demo for running a CodeMirror parser over a piece of code without
// creating an actual editor.
(function(){
function normaliseString(string) {
var tab = "";
for (var i = 0; i &lt; indentUnit; i++) tab += " ";
string = string.replace(/\t/g, tab).replace(/\u00a0/g, " ").replace(/\r\n?/g, "\n");
var pos = 0, parts = [], lines = string.split("\n");
for (var line = 0; line &lt; lines.length; line++) {
if (line != 0) parts.push("\n");
parts.push(lines[line]);
}
return {
next: function() {
if (pos &lt; parts.length) return parts[pos++];
else throw StopIteration;
}
};
}
window.highlightText = function(string, output, parser) {
var parser = (parser || Editor.Parser).make(stringStream(normaliseString(string)));
try {
while (true) {
var token = parser.next();
var span = document.createElement("SPAN");
span.className = token.style;
span.appendChild(document.createTextNode(token.value));
output.appendChild(span);
}
}
catch (e) {
if (e != StopIteration) throw e;
}
}
})();
</textarea>
</div>
<button onclick="highlight()">Run highlighter</button>
<div>
<div id="numbers" style="float: left; width: 2em; margin-right: .5em; text-align: right; font-family: monospace; color: #CCC;"></div>
<pre id="output" style="font-family: monospace"></pre>
</div>
<script type="text/javascript">
// Simple hack to demonstrate adding line numbers. Just pass the DOM node as
// the second argument to highlightText when you don't need those
function highlight() {
var lineNo = 1, output = document.getElementById("output"), numbers = document.getElementById("numbers");
output.innerHTML = numbers.innerHTML = "";
function addLine(line) {
numbers.appendChild(document.createTextNode(String(lineNo++)));
numbers.appendChild(document.createElement("BR"));
for (var i = 0; i < line.length; i++) output.appendChild(line[i]);
output.appendChild(document.createElement("BR"));
}
highlightText(document.getElementById("code").value, addLine);
}
</script>
</body>
</html>

View File

@ -0,0 +1,298 @@
/* CodeMirror main module
*
* Implements the CodeMirror constructor and prototype, which take care
* of initializing the editor frame, and providing the outside interface.
*/
// The CodeMirrorConfig object is used to specify a default
// configuration. If you specify such an object before loading this
// file, the values you put into it will override the defaults given
// below. You can also assign to it after loading.
var CodeMirrorConfig = window.CodeMirrorConfig || {};
var CodeMirror = (function(){
function setDefaults(object, defaults) {
for (var option in defaults) {
if (!object.hasOwnProperty(option))
object[option] = defaults[option];
}
}
function forEach(array, action) {
for (var i = 0; i < array.length; i++)
action(array[i]);
}
// These default options can be overridden by passing a set of
// options to a specific CodeMirror constructor. See manual.html for
// their meaning.
setDefaults(CodeMirrorConfig, {
stylesheet: "",
path: "",
parserfile: [],
basefiles: ["util.js", "stringstream.js", "select.js", "undo.js", "editor.js", "tokenize.js"],
iframeClass: null,
passDelay: 200,
passTime: 50,
continuousScanning: false,
saveFunction: null,
onChange: null,
undoDepth: 50,
undoDelay: 800,
disableSpellcheck: true,
textWrapping: true,
readOnly: false,
width: "100%",
height: "300px",
autoMatchParens: false,
parserConfig: null,
tabMode: "indent", // or "spaces", "default", "shift"
reindentOnLoad: false,
activeTokens: null,
cursorActivity: null,
lineNumbers: false,
indentUnit: 2
});
function wrapLineNumberDiv(place) {
return function(node) {
var container = document.createElement("DIV"),
nums = document.createElement("DIV"),
scroller = document.createElement("DIV");
container.style.position = "relative";
nums.style.position = "absolute";
nums.style.height = "100%";
if (nums.style.setExpression) {
try {nums.style.setExpression("height", "this.previousSibling.offsetHeight + 'px'");}
catch(e) {} // Seems to throw 'Not Implemented' on some IE8 versions
}
nums.style.top = "0px";
nums.style.overflow = "hidden";
place(container);
container.appendChild(node);
container.appendChild(nums);
scroller.className = "CodeMirror-line-numbers";
nums.appendChild(scroller);
}
}
function applyLineNumbers(frame) {
var win = frame.contentWindow, doc = win.document,
nums = frame.nextSibling, scroller = nums.firstChild;
var nextNum = 1, barWidth = null;
function sizeBar() {
if (nums.offsetWidth != barWidth) {
barWidth = nums.offsetWidth;
nums.style.left = "-" + (frame.parentNode.style.marginLeft = barWidth + "px");
}
}
function update() {
var diff = 20 + Math.max(doc.body.offsetHeight, frame.offsetHeight) - scroller.offsetHeight;
for (var n = Math.ceil(diff / 10); n > 0; n--) {
var div = document.createElement("DIV");
div.appendChild(document.createTextNode(nextNum++));
scroller.appendChild(div);
}
nums.scrollTop = doc.body.scrollTop || doc.documentElement.scrollTop || 0;
}
sizeBar();
update();
win.addEventHandler(win, "scroll", update);
setInterval(sizeBar, 500);
}
function CodeMirror(place, options) {
// Backward compatibility for deprecated options.
if (options.dumbTabs) options.tabMode = "spaces";
else if (options.normalTab) options.tabMode = "default";
// Use passed options, if any, to override defaults.
this.options = options = options || {};
setDefaults(options, CodeMirrorConfig);
var frame = this.frame = document.createElement("IFRAME");
if (options.iframeClass) frame.className = options.iframeClass;
frame.frameBorder = 0;
frame.src = "javascript:false;";
frame.style.border = "0";
frame.style.width = options.width;
frame.style.height = options.height;
// display: block occasionally suppresses some Firefox bugs, so we
// always add it, redundant as it sounds.
frame.style.display = "block";
if (place.appendChild) {
var node = place;
place = function(n){node.appendChild(n);};
}
if (options.lineNumbers) place = wrapLineNumberDiv(place);
place(frame);
// Link back to this object, so that the editor can fetch options
// and add a reference to itself.
frame.CodeMirror = this;
this.win = frame.contentWindow;
if (typeof options.parserfile == "string")
options.parserfile = [options.parserfile];
if (typeof options.stylesheet == "string")
options.stylesheet = [options.stylesheet];
var html = ["<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.0 Transitional//EN\" \"http://www.w3.org/TR/html4/loose.dtd\"><html><head>"];
forEach(options.stylesheet, function(file) {
html.push("<link rel=\"stylesheet\" type=\"text/css\" href=\"" + file + "\"/>");
});
forEach(options.basefiles.concat(options.parserfile), function(file) {
html.push("<script type=\"text/javascript\" src=\"" + options.path + file + "\"></script>");
});
html.push("</head><body style=\"border-width: 0;\" class=\"editbox\" spellcheck=\"" +
(options.disableSpellcheck ? "false" : "true") + "\"></body></html>");
var doc = this.win.document;
doc.open();
doc.write(html.join(""));
doc.close();
}
CodeMirror.prototype = {
init: function() {
if (this.options.initCallback) this.options.initCallback(this);
if (this.options.lineNumbers) applyLineNumbers(this.frame);
if (this.options.reindentOnLoad) this.reindent();
},
getCode: function() {return this.editor.getCode();},
setCode: function(code) {this.editor.importCode(code);},
selection: function() {return this.editor.selectedText();},
reindent: function() {this.editor.reindent();},
reindentSelection: function() {this.editor.reindentSelection(null);},
focus: function() {
this.win.focus();
if (this.editor.selectionSnapshot) // IE hack
this.win.select.selectCoords(this.win, this.editor.selectionSnapshot);
},
replaceSelection: function(text) {
this.focus();
this.editor.replaceSelection(text);
return true;
},
replaceChars: function(text, start, end) {
this.editor.replaceChars(text, start, end);
},
getSearchCursor: function(string, fromCursor) {
return this.editor.getSearchCursor(string, fromCursor);
},
undo: function() {this.editor.history.undo();},
redo: function() {this.editor.history.redo();},
historySize: function() {return this.editor.history.historySize();},
clearHistory: function() {this.editor.history.clear();},
grabKeys: function(callback, filter) {this.editor.grabKeys(callback, filter);},
ungrabKeys: function() {this.editor.ungrabKeys();},
setParser: function(name) {this.editor.setParser(name);},
cursorPosition: function(start) {
if (this.win.select.ie_selection) this.focus();
return this.editor.cursorPosition(start);
},
firstLine: function() {return this.editor.firstLine();},
lastLine: function() {return this.editor.lastLine();},
nextLine: function(line) {return this.editor.nextLine(line);},
prevLine: function(line) {return this.editor.prevLine(line);},
lineContent: function(line) {return this.editor.lineContent(line);},
setLineContent: function(line, content) {this.editor.setLineContent(line, content);},
insertIntoLine: function(line, position, content) {this.editor.insertIntoLine(line, position, content);},
selectLines: function(startLine, startOffset, endLine, endOffset) {
this.win.focus();
this.editor.selectLines(startLine, startOffset, endLine, endOffset);
},
nthLine: function(n) {
var line = this.firstLine();
for (; n > 1 && line !== false; n--)
line = this.nextLine(line);
return line;
},
lineNumber: function(line) {
var num = 0;
while (line !== false) {
num++;
line = this.prevLine(line);
}
return num;
},
// Old number-based line interface
jumpToLine: function(n) {
this.selectLines(this.nthLine(n), 0);
this.win.focus();
},
currentLine: function() {
return this.lineNumber(this.cursorPosition().line);
}
};
CodeMirror.InvalidLineHandle = {toString: function(){return "CodeMirror.InvalidLineHandle";}};
CodeMirror.replace = function(element) {
if (typeof element == "string")
element = document.getElementById(element);
return function(newElement) {
element.parentNode.replaceChild(newElement, element);
};
};
CodeMirror.fromTextArea = function(area, options) {
if (typeof area == "string")
area = document.getElementById(area);
options = options || {};
if (area.style.width && options.width == null)
options.width = area.style.width;
if (area.style.height && options.height == null)
options.height = area.style.height;
if (options.content == null) options.content = area.value;
if (area.form) {
function updateField() {
area.value = mirror.getCode();
}
if (typeof area.form.addEventListener == "function")
area.form.addEventListener("submit", updateField, false);
else
area.form.attachEvent("onsubmit", updateField);
}
function insert(frame) {
if (area.nextSibling)
area.parentNode.insertBefore(frame, area.nextSibling);
else
area.parentNode.appendChild(frame);
}
area.style.display = "none";
var mirror = new CodeMirror(insert, options);
return mirror;
};
CodeMirror.isProbablySupported = function() {
// This is rather awful, but can be useful.
var match;
if (window.opera)
return Number(window.opera.version()) >= 9.52;
else if (/Apple Computers, Inc/.test(navigator.vendor) && (match = navigator.userAgent.match(/Version\/(\d+(?:\.\d+)?)\./)))
return Number(match[1]) >= 3;
else if (document.selection && window.ActiveXObject && (match = navigator.userAgent.match(/MSIE (\d+(?:\.\d*)?)\b/)))
return Number(match[1]) >= 6;
else if (match = navigator.userAgent.match(/gecko\/(\d{8})/i))
return Number(match[1]) >= 20050901;
else if (match = navigator.userAgent.match(/AppleWebKit\/(\d+)/))
return Number(match[1]) >= 525;
else
return null;
};
return CodeMirror;
})();

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,68 @@
// Minimal framing needed to use CodeMirror-style parsers to highlight
// code. Load this along with tokenize.js, stringstream.js, and your
// parser. Then call highlightText, passing a string as the first
// argument, and as the second argument either a callback function
// that will be called with an array of SPAN nodes for every line in
// the code, or a DOM node to which to append these spans, and
// optionally (not needed if you only loaded one parser) a parser
// object.
// Stuff from util.js that the parsers are using.
var StopIteration = {toString: function() {return "StopIteration"}};
var Editor = {};
var indentUnit = 2;
(function(){
function normaliseString(string) {
var tab = "";
for (var i = 0; i < indentUnit; i++) tab += " ";
string = string.replace(/\t/g, tab).replace(/\u00a0/g, " ").replace(/\r\n?/g, "\n");
var pos = 0, parts = [], lines = string.split("\n");
for (var line = 0; line < lines.length; line++) {
if (line != 0) parts.push("\n");
parts.push(lines[line]);
}
return {
next: function() {
if (pos < parts.length) return parts[pos++];
else throw StopIteration;
}
};
}
window.highlightText = function(string, callback, parser) {
var parser = (parser || Editor.Parser).make(stringStream(normaliseString(string)));
var line = [];
if (callback.nodeType == 1) {
var node = callback;
callback = function(line) {
for (var i = 0; i < line.length; i++)
node.appendChild(line[i]);
node.appendChild(document.createElement("BR"));
};
}
try {
while (true) {
var token = parser.next();
if (token.value == "\n") {
callback(line);
line = [];
}
else {
var span = document.createElement("SPAN");
span.className = token.style;
span.appendChild(document.createTextNode(token.value));
line.push(span);
}
}
}
catch (e) {
if (e != StopIteration) throw e;
}
if (line.length) callback(line);
}
})();

View File

@ -0,0 +1,81 @@
/* Demonstration of embedding CodeMirror in a bigger application. The
* interface defined here is a mess of prompts and confirms, and
* should probably not be used in a real project.
*/
function MirrorFrame(place, options) {
this.home = document.createElement("DIV");
if (place.appendChild)
place.appendChild(this.home);
else
place(this.home);
var self = this;
function makeButton(name, action) {
var button = document.createElement("INPUT");
button.type = "button";
button.value = name;
self.home.appendChild(button);
button.onclick = function(){self[action].call(self);};
}
makeButton("Search", "search");
makeButton("Replace", "replace");
makeButton("Current line", "line");
makeButton("Jump to line", "jump");
makeButton("Insert constructor", "macro");
makeButton("Indent all", "reindent");
this.mirror = new CodeMirror(this.home, options);
}
MirrorFrame.prototype = {
search: function() {
var text = prompt("Enter search term:", "");
if (!text) return;
var first = true;
do {
var cursor = this.mirror.getSearchCursor(text, first);
first = false;
while (cursor.findNext()) {
cursor.select();
if (!confirm("Search again?"))
return;
}
} while (confirm("End of document reached. Start over?"));
},
replace: function() {
// This is a replace-all, but it is possible to implement a
// prompting replace.
var from = prompt("Enter search string:", ""), to;
if (from) to = prompt("What should it be replaced with?", "");
if (to == null) return;
var cursor = this.mirror.getSearchCursor(from, false);
while (cursor.findNext())
cursor.replace(to);
},
jump: function() {
var line = prompt("Jump to line:", "");
if (line && !isNaN(Number(line)))
this.mirror.jumpToLine(Number(line));
},
line: function() {
alert("The cursor is currently at line " + this.mirror.currentLine());
this.mirror.focus();
},
macro: function() {
var name = prompt("Name your constructor:", "");
if (name)
this.mirror.replaceSelection("function " + name + "() {\n \n}\n\n" + name + ".prototype = {\n \n};\n");
},
reindent: function() {
this.mirror.reindent();
}
};

View File

@ -0,0 +1,155 @@
/* Simple parser for CSS */
var CSSParser = Editor.Parser = (function() {
var tokenizeCSS = (function() {
function normal(source, setState) {
var ch = source.next();
if (ch == "@") {
source.nextWhileMatches(/\w/);
return "css-at";
}
else if (ch == "/" && source.equals("*")) {
setState(inCComment);
return null;
}
else if (ch == "<" && source.equals("!")) {
setState(inSGMLComment);
return null;
}
else if (ch == "=") {
return "css-compare";
}
else if (source.equals("=") && (ch == "~" || ch == "|")) {
source.next();
return "css-compare";
}
else if (ch == "\"" || ch == "'") {
setState(inString(ch));
return null;
}
else if (ch == "#") {
source.nextWhileMatches(/\w/);
return "css-hash";
}
else if (ch == "!") {
source.nextWhileMatches(/[ \t]/);
source.nextWhileMatches(/\w/);
return "css-important";
}
else if (/\d/.test(ch)) {
source.nextWhileMatches(/[\w.%]/);
return "css-unit";
}
else if (/[,.+>*\/]/.test(ch)) {
return "css-select-op";
}
else if (/[;{}:\[\]]/.test(ch)) {
return "css-punctuation";
}
else {
source.nextWhileMatches(/[\w\\\-_]/);
return "css-identifier";
}
}
function inCComment(source, setState) {
var maybeEnd = false;
while (!source.endOfLine()) {
var ch = source.next();
if (maybeEnd && ch == "/") {
setState(normal);
break;
}
maybeEnd = (ch == "*");
}
return "css-comment";
}
function inSGMLComment(source, setState) {
var dashes = 0;
while (!source.endOfLine()) {
var ch = source.next();
if (dashes >= 2 && ch == ">") {
setState(normal);
break;
}
dashes = (ch == "-") ? dashes + 1 : 0;
}
return "css-comment";
}
function inString(quote) {
return function(source, setState) {
var escaped = false;
while (!source.endOfLine()) {
var ch = source.next();
if (ch == quote && !escaped)
break;
escaped = !escaped && ch == "\\";
}
if (!escaped)
setState(normal);
return "css-string";
};
}
return function(source, startState) {
return tokenizer(source, startState || normal);
};
})();
function indentCSS(inBraces, inRule, base) {
return function(nextChars) {
if (!inBraces || /^\}/.test(nextChars)) return base;
else if (inRule) return base + indentUnit * 2;
else return base + indentUnit;
};
}
// This is a very simplistic parser -- since CSS does not really
// nest, it works acceptably well, but some nicer colouroing could
// be provided with a more complicated parser.
function parseCSS(source, basecolumn) {
basecolumn = basecolumn || 0;
var tokens = tokenizeCSS(source);
var inBraces = false, inRule = false;
var iter = {
next: function() {
var token = tokens.next(), style = token.style, content = token.content;
if (style == "css-identifier" && inRule)
token.style = "css-value";
if (style == "css-hash")
token.style = inRule ? "css-colorcode" : "css-identifier";
if (content == "\n")
token.indentation = indentCSS(inBraces, inRule, basecolumn);
if (content == "{")
inBraces = true;
else if (content == "}")
inBraces = inRule = false;
else if (inBraces && content == ";")
inRule = false;
else if (inBraces && style != "css-comment" && style != "whitespace")
inRule = true;
return token;
},
copy: function() {
var _inBraces = inBraces, _inRule = inRule, _tokenState = tokens.state;
return function(source) {
tokens = tokenizeCSS(source, _tokenState);
inBraces = _inBraces;
inRule = _inRule;
return iter;
};
}
};
return iter;
}
return {make: parseCSS, electricChars: "}"};
})();

View File

@ -0,0 +1,32 @@
var DummyParser = Editor.Parser = (function() {
function tokenizeDummy(source) {
while (!source.endOfLine()) source.next();
return "text";
}
function parseDummy(source) {
function indentTo(n) {return function() {return n;}}
source = tokenizer(source, tokenizeDummy);
var space = 0;
var iter = {
next: function() {
var tok = source.next();
if (tok.type == "whitespace") {
if (tok.value == "\n") tok.indentation = indentTo(space);
else space = tok.value.length;
}
return tok;
},
copy: function() {
var _space = space;
return function(_source) {
space = _space;
source = tokenizer(_source, tokenizeDummy);
return iter;
};
}
};
return iter;
}
return {make: parseDummy};
})();

View File

@ -0,0 +1,74 @@
var HTMLMixedParser = Editor.Parser = (function() {
if (!(CSSParser && JSParser && XMLParser))
throw new Error("CSS, JS, and XML parsers must be loaded for HTML mixed mode to work.");
XMLParser.configure({useHTMLKludges: true});
function parseMixed(stream) {
var htmlParser = XMLParser.make(stream), localParser = null, inTag = false;
var iter = {next: top, copy: copy};
function top() {
var token = htmlParser.next();
if (token.content == "<")
inTag = true;
else if (token.style == "xml-tagname" && inTag === true)
inTag = token.content.toLowerCase();
else if (token.content == ">") {
if (inTag == "script")
iter.next = local(JSParser, "</script");
else if (inTag == "style")
iter.next = local(CSSParser, "</style");
inTag = false;
}
return token;
}
function local(parser, tag) {
var baseIndent = htmlParser.indentation();
localParser = parser.make(stream, baseIndent + indentUnit);
return function() {
if (stream.lookAhead(tag, false, false, true)) {
localParser = null;
iter.next = top;
return top();
}
var token = localParser.next();
var lt = token.value.lastIndexOf("<"), sz = Math.min(token.value.length - lt, tag.length);
if (lt != -1 && token.value.slice(lt, lt + sz).toLowerCase() == tag.slice(0, sz) &&
stream.lookAhead(tag.slice(sz), false, false, true)) {
stream.push(token.value.slice(lt));
token.value = token.value.slice(0, lt);
}
if (token.indentation) {
var oldIndent = token.indentation;
token.indentation = function(chars) {
if (chars == "</")
return baseIndent;
else
return oldIndent(chars);
}
}
return token;
};
}
function copy() {
var _html = htmlParser.copy(), _local = localParser && localParser.copy(),
_next = iter.next, _inTag = inTag;
return function(_stream) {
stream = _stream;
htmlParser = _html(_stream);
localParser = _local && _local(_stream);
iter.next = _next;
inTag = _inTag;
return iter;
};
}
return iter;
}
return {make: parseMixed, electricChars: "{}/:"};
})();

View File

@ -0,0 +1,341 @@
/* Parse function for JavaScript. Makes use of the tokenizer from
* tokenizejavascript.js. Note that your parsers do not have to be
* this complicated -- if you don't want to recognize local variables,
* in many languages it is enough to just look for braces, semicolons,
* parentheses, etc, and know when you are inside a string or comment.
*
* See manual.html for more info about the parser interface.
*/
var JSParser = Editor.Parser = (function() {
// Token types that can be considered to be atoms.
var atomicTypes = {"atom": true, "number": true, "variable": true, "string": true, "regexp": true};
// Constructor for the lexical context objects.
function JSLexical(indented, column, type, align, prev, info) {
// indentation at start of this line
this.indented = indented;
// column at which this scope was opened
this.column = column;
// type of scope ('vardef', 'stat' (statement), 'form' (special form), '[', '{', or '(')
this.type = type;
// '[', '{', or '(' blocks that have any text after their opening
// character are said to be 'aligned' -- any lines below are
// indented all the way to the opening character.
if (align != null)
this.align = align;
// Parent scope, if any.
this.prev = prev;
this.info = info;
}
// My favourite JavaScript indentation rules.
function indentJS(lexical) {
return function(firstChars) {
var firstChar = firstChars && firstChars.charAt(0), type = lexical.type;
var closing = firstChar == type;
if (type == "vardef")
return lexical.indented + 4;
else if (type == "form" && firstChar == "{")
return lexical.indented;
else if (type == "stat" || type == "form")
return lexical.indented + indentUnit;
else if (lexical.info == "switch" && !closing)
return lexical.indented + (/^(?:case|default)\b/.test(firstChars) ? indentUnit : 2 * indentUnit);
else if (lexical.align)
return lexical.column - (closing ? 1 : 0);
else
return lexical.indented + (closing ? 0 : indentUnit);
};
}
// The parser-iterator-producing function itself.
function parseJS(input, basecolumn) {
// Wrap the input in a token stream
var tokens = tokenizeJavaScript(input);
// The parser state. cc is a stack of actions that have to be
// performed to finish the current statement. For example we might
// know that we still need to find a closing parenthesis and a
// semicolon. Actions at the end of the stack go first. It is
// initialized with an infinitely looping action that consumes
// whole statements.
var cc = [statements];
// Context contains information about the current local scope, the
// variables defined in that, and the scopes above it.
var context = null;
// The lexical scope, used mostly for indentation.
var lexical = new JSLexical((basecolumn || 0) - indentUnit, 0, "block", false);
// Current column, and the indentation at the start of the current
// line. Used to create lexical scope objects.
var column = 0;
var indented = 0;
// Variables which are used by the mark, cont, and pass functions
// below to communicate with the driver loop in the 'next'
// function.
var consume, marked;
// The iterator object.
var parser = {next: next, copy: copy};
function next(){
// Start by performing any 'lexical' actions (adjusting the
// lexical variable), or the operations below will be working
// with the wrong lexical state.
while(cc[cc.length - 1].lex)
cc.pop()();
// Fetch a token.
var token = tokens.next();
// Adjust column and indented.
if (token.type == "whitespace" && column == 0)
indented = token.value.length;
column += token.value.length;
if (token.content == "\n"){
indented = column = 0;
// If the lexical scope's align property is still undefined at
// the end of the line, it is an un-aligned scope.
if (!("align" in lexical))
lexical.align = false;
// Newline tokens get an indentation function associated with
// them.
token.indentation = indentJS(lexical);
}
// No more processing for meaningless tokens.
if (token.type == "whitespace" || token.type == "comment")
return token;
// When a meaningful token is found and the lexical scope's
// align is undefined, it is an aligned scope.
if (!("align" in lexical))
lexical.align = true;
// Execute actions until one 'consumes' the token and we can
// return it.
while(true) {
consume = marked = false;
// Take and execute the topmost action.
cc.pop()(token.type, token.content);
if (consume){
// Marked is used to change the style of the current token.
if (marked)
token.style = marked;
// Here we differentiate between local and global variables.
else if (token.type == "variable" && inScope(token.content))
token.style = "js-localvariable";
return token;
}
}
}
// This makes a copy of the parser state. It stores all the
// stateful variables in a closure, and returns a function that
// will restore them when called with a new input stream. Note
// that the cc array has to be copied, because it is contantly
// being modified. Lexical objects are not mutated, and context
// objects are not mutated in a harmful way, so they can be shared
// between runs of the parser.
function copy(){
var _context = context, _lexical = lexical, _cc = cc.concat([]), _tokenState = tokens.state;
return function copyParser(input){
context = _context;
lexical = _lexical;
cc = _cc.concat([]); // copies the array
column = indented = 0;
tokens = tokenizeJavaScript(input, _tokenState);
return parser;
};
}
// Helper function for pushing a number of actions onto the cc
// stack in reverse order.
function push(fs){
for (var i = fs.length - 1; i >= 0; i--)
cc.push(fs[i]);
}
// cont and pass are used by the action functions to add other
// actions to the stack. cont will cause the current token to be
// consumed, pass will leave it for the next action.
function cont(){
push(arguments);
consume = true;
}
function pass(){
push(arguments);
consume = false;
}
// Used to change the style of the current token.
function mark(style){
marked = style;
}
// Push a new scope. Will automatically link the current scope.
function pushcontext(){
context = {prev: context, vars: {"this": true, "arguments": true}};
}
// Pop off the current scope.
function popcontext(){
context = context.prev;
}
// Register a variable in the current scope.
function register(varname){
if (context){
mark("js-variabledef");
context.vars[varname] = true;
}
}
// Check whether a variable is defined in the current scope.
function inScope(varname){
var cursor = context;
while (cursor) {
if (cursor.vars[varname])
return true;
cursor = cursor.prev;
}
return false;
}
// Push a new lexical context of the given type.
function pushlex(type, info) {
var result = function(){
lexical = new JSLexical(indented, column, type, null, lexical, info)
};
result.lex = true;
return result;
}
// Pop off the current lexical context.
function poplex(){
lexical = lexical.prev;
}
poplex.lex = true;
// The 'lex' flag on these actions is used by the 'next' function
// to know they can (and have to) be ran before moving on to the
// next token.
// Creates an action that discards tokens until it finds one of
// the given type.
function expect(wanted){
return function expecting(type){
if (type == wanted) cont();
else cont(arguments.callee);
};
}
// Looks for a statement, and then calls itself.
function statements(type){
return pass(statement, statements);
}
// Dispatches various types of statements based on the type of the
// current token.
function statement(type){
if (type == "var") cont(pushlex("vardef"), vardef1, expect(";"), poplex);
else if (type == "keyword a") cont(pushlex("form"), expression, statement, poplex);
else if (type == "keyword b") cont(pushlex("form"), statement, poplex);
else if (type == "{") cont(pushlex("}"), block, poplex);
else if (type == "function") cont(functiondef);
else if (type == "for") cont(pushlex("form"), expect("("), pushlex(")"), forspec1, expect(")"), poplex, statement, poplex);
else if (type == "variable") cont(pushlex("stat"), maybelabel);
else if (type == "switch") cont(pushlex("form"), expression, pushlex("}", "switch"), expect("{"), block, poplex, poplex);
else if (type == "case") cont(expression, expect(":"));
else if (type == "default") cont(expect(":"));
else if (type == "catch") cont(pushlex("form"), pushcontext, expect("("), funarg, expect(")"), statement, poplex, popcontext);
else pass(pushlex("stat"), expression, expect(";"), poplex);
}
// Dispatch expression types.
function expression(type){
if (atomicTypes.hasOwnProperty(type)) cont(maybeoperator);
else if (type == "function") cont(functiondef);
else if (type == "keyword c") cont(expression);
else if (type == "(") cont(pushlex(")"), expression, expect(")"), poplex, maybeoperator);
else if (type == "operator") cont(expression);
else if (type == "[") cont(pushlex("]"), commasep(expression, "]"), poplex, maybeoperator);
else if (type == "{") cont(pushlex("}"), commasep(objprop, "}"), poplex, maybeoperator);
}
// Called for places where operators, function calls, or
// subscripts are valid. Will skip on to the next action if none
// is found.
function maybeoperator(type){
if (type == "operator") cont(expression);
else if (type == "(") cont(pushlex(")"), expression, commasep(expression, ")"), poplex, maybeoperator);
else if (type == ".") cont(property, maybeoperator);
else if (type == "[") cont(pushlex("]"), expression, expect("]"), poplex, maybeoperator);
}
// When a statement starts with a variable name, it might be a
// label. If no colon follows, it's a regular statement.
function maybelabel(type){
if (type == ":") cont(poplex, statement);
else pass(maybeoperator, expect(";"), poplex);
}
// Property names need to have their style adjusted -- the
// tokenizer thinks they are variables.
function property(type){
if (type == "variable") {mark("js-property"); cont();}
}
// This parses a property and its value in an object literal.
function objprop(type){
if (type == "variable") mark("js-property");
if (atomicTypes.hasOwnProperty(type)) cont(expect(":"), expression);
}
// Parses a comma-separated list of the things that are recognized
// by the 'what' argument.
function commasep(what, end){
function proceed(type) {
if (type == ",") cont(what, proceed);
else if (type == end) cont();
else cont(expect(end));
};
return function commaSeparated(type) {
if (type == end) cont();
else pass(what, proceed);
};
}
// Look for statements until a closing brace is found.
function block(type){
if (type == "}") cont();
else pass(statement, block);
}
// Variable definitions are split into two actions -- 1 looks for
// a name or the end of the definition, 2 looks for an '=' sign or
// a comma.
function vardef1(type, value){
if (type == "variable"){register(value); cont(vardef2);}
else cont();
}
function vardef2(type, value){
if (value == "=") cont(expression, vardef2);
else if (type == ",") cont(vardef1);
}
// For loops.
function forspec1(type){
if (type == "var") cont(vardef1, forspec2);
else if (type == ";") pass(forspec2);
else if (type == "variable") cont(formaybein);
else pass(forspec2);
}
function formaybein(type, value){
if (value == "in") cont(expression);
else cont(maybeoperator, forspec2);
}
function forspec2(type, value){
if (type == ";") cont(forspec3);
else if (value == "in") cont(expression);
else cont(expression, expect(";"), forspec3);
}
function forspec3(type) {
if (type == ")") pass();
else cont(expression);
}
// A function definition creates a new context, and the variables
// in its argument list have to be added to this context.
function functiondef(type, value){
if (type == "variable"){register(value); cont(functiondef);}
else if (type == "(") cont(pushcontext, commasep(funarg, ")"), statement, popcontext);
}
function funarg(type, value){
if (type == "variable"){register(value); cont();}
}
return parser;
}
return {make: parseJS, electricChars: "{}:"};
})();

View File

@ -0,0 +1,162 @@
var SparqlParser = Editor.Parser = (function() {
function wordRegexp(words) {
return new RegExp("^(?:" + words.join("|") + ")$", "i");
}
var ops = wordRegexp(["str", "lang", "langmatches", "datatype", "bound", "sameterm", "isiri", "isuri",
"isblank", "isliteral", "union", "a"]);
var keywords = wordRegexp(["base", "prefix", "select", "distinct", "reduced", "construct", "describe",
"ask", "from", "named", "where", "order", "limit", "offset", "filter", "optional",
"graph", "by", "asc", "desc", ]);
var operatorChars = /[*+\-<>=&|]/;
var tokenizeSparql = (function() {
function normal(source, setState) {
var ch = source.next();
if (ch == "$" || ch == "?") {
source.nextWhileMatches(/[\w\d]/);
return "sp-var";
}
else if (ch == "<" && !source.matches(/[\s\u00a0=]/)) {
source.nextWhileMatches(/[^\s\u00a0>]/);
if (source.equals(">")) source.next();
return "sp-uri";
}
else if (ch == "\"" || ch == "'") {
setState(inLiteral(ch));
return null;
}
else if (/[{}\(\),\.;\[\]]/.test(ch)) {
return "sp-punc";
}
else if (ch == "#") {
while (!source.endOfLine()) source.next();
return "sp-comment";
}
else if (operatorChars.test(ch)) {
source.nextWhileMatches(operatorChars);
return "sp-operator";
}
else if (ch == ":") {
source.nextWhileMatches(/[\w\d\._\-]/);
return "sp-prefixed";
}
else {
source.nextWhileMatches(/[_\w\d]/);
if (source.equals(":")) {
source.next();
source.nextWhileMatches(/[\w\d_\-]/);
return "sp-prefixed";
}
var word = source.get(), type;
if (ops.test(word))
type = "sp-operator";
else if (keywords.test(word))
type = "sp-keyword";
else
type = "sp-word";
return {style: type, content: word};
}
}
function inLiteral(quote) {
return function(source, setState) {
var escaped = false;
while (!source.endOfLine()) {
var ch = source.next();
if (ch == quote && !escaped) {
setState(normal);
break;
}
escaped = !escaped && ch == "\\";
}
return "sp-literal";
};
}
return function(source, startState) {
return tokenizer(source, startState || normal);
};
})();
function indentSparql(context) {
return function(nextChars) {
var firstChar = nextChars && nextChars.charAt(0);
if (/[\]\}]/.test(firstChar))
while (context && context.type == "pattern") context = context.prev;
var closing = context && firstChar == matching[context.type];
if (!context)
return 0;
else if (context.type == "pattern")
return context.col;
else if (context.align)
return context.col - (closing ? context.width : 0);
else
return context.indent + (closing ? 0 : indentUnit);
}
}
function parseSparql(source) {
var tokens = tokenizeSparql(source);
var context = null, indent = 0, col = 0;
function pushContext(type, width) {
context = {prev: context, indent: indent, col: col, type: type, width: width};
}
function popContext() {
context = context.prev;
}
var iter = {
next: function() {
var token = tokens.next(), type = token.style, content = token.content, width = token.value.length;
if (content == "\n") {
token.indentation = indentSparql(context);
indent = col = 0;
if (context && context.align == null) context.align = false;
}
else if (type == "whitespace" && col == 0) {
indent = width;
}
else if (type != "sp-comment" && context && context.align == null) {
context.align = true;
}
if (content != "\n") col += width;
if (/[\[\{\(]/.test(content)) {
pushContext(content, width);
}
else if (/[\]\}\)]/.test(content)) {
while (context && context.type == "pattern")
popContext();
if (context && content == matching[context.type])
popContext();
}
else if (content == "." && context && context.type == "pattern") {
popContext();
}
else if ((type == "sp-word" || type == "sp-prefixed" || type == "sp-uri" || type == "sp-var" || type == "sp-literal") &&
context && /[\{\[]/.test(context.type)) {
pushContext("pattern", width);
}
return token;
},
copy: function() {
var _context = context, _indent = indent, _col = col, _tokenState = tokens.state;
return function(source) {
tokens = tokenizeSparql(source, _tokenState);
context = _context;
indent = _indent;
col = _col;
return iter;
};
}
};
return iter;
}
return {make: parseSparql, electricChars: "}]"};
})();

View File

@ -0,0 +1,107 @@
/* Simple parser for Survex files (based on the CSS example) */
// The tokenizer breaks up the text into convincing chunks (I think the white-space is parser automatically)
var SVXParser = Editor.Parser = (function() {
var tokenizeSVX = (function() {
function normal(source, setState) {
var ch = source.next();
if (ch == ";") {
source.nextWhile(matcher(/[^\n]/));
return "svx-comment";
}
else if (ch == "*") {
source.nextWhile(matcher(/\w/));
return "svx-command";
}
else if (ch == "\"" || ch == "'") {
var escaped = false;
while (!source.endOfLine()) {
var nch = source.next();
if (nch == ch && !escaped)
break;
escaped = !escaped && nch == "\\";
}
return "svx-string";
}
else if (/[\d\-+.]/.test(ch)) {
source.nextWhile(matcher(/[\d.]/));
return "svx-measure";
}
else {
source.nextWhile(matcher(/\S/));
return "svx-word";
}
}
return function(source, startState) {
return tokenizer(source, startState || normal);
};
})();
// survex doesn't have indentation; but you get double linefeeds if you leave this out.
function indentSVX() {
return function(nextChars) {
return 0;
};
}
// Then this simple parser fixes up the obvious errors made by the tokenizer (which could only operate on characters)
// A very fancy upgrade could make it capable of handling the *data commands which make it accept different orderings of
// the parameters -- though this may be a challenge because the whole file needs reparsing when that happens -- don't
// know how optimized the basic code is to be able to call for such to happen when a formatting command like this changes.
function parseSVX(source, basecolumn) {
basecolumn = basecolumn || 0;
var tokens = tokenizeSVX(source);
var inCommand = false;
var ntokeninline = -1;
var iter = {
next: function() {
var token = tokens.next(), style = token.style, content = token.content;
if (content == "\n") {
ntokeninline = -1;
inCommand = false;
token.indentation = indentSVX();
}
else if (style != "whitespace")
ntokeninline += 1;
if (style == "svx-command") {
inCommand = (ntokeninline == 0);
if (!inCommand)
token.style = "svx-word";
else if (content == "*begin")
token.style = "svx-begin";
else if (content == "*end")
token.style = "svx-end";
}
if (!inCommand && style == "svx-measure") {
if (ntokeninline < 2)
token.style = "svx-word";
}
if (!inCommand && style == "svx-word" && (ntokeninline == 4)) {
if (content == "down" || content == "up")
token.style = "svx-measure";
}
return token;
},
copy: function() {
var _inCommand = inCommand, _tokenState = tokens.state, _ntokeninline = ntokeninline;
return function(source) {
tokens = tokenizeSVX(source, _tokenState);
inCommand = _inCommand;
ntokeninline = _ntokeninline;
return iter;
};
}
};
return iter;
}
return {make: parseSVX};
})();

View File

@ -0,0 +1,292 @@
/* This file defines an XML parser, with a few kludges to make it
* useable for HTML. autoSelfClosers defines a set of tag names that
* are expected to not have a closing tag, and doNotIndent specifies
* the tags inside of which no indentation should happen (see Config
* object). These can be disabled by passing the editor an object like
* {useHTMLKludges: false} as parserConfig option.
*/
var XMLParser = Editor.Parser = (function() {
var Kludges = {
autoSelfClosers: {"br": true, "img": true, "hr": true, "link": true, "input": true,
"meta": true, "col": true, "frame": true, "base": true, "area": true},
doNotIndent: {"pre": true, "!cdata": true}
};
var NoKludges = {autoSelfClosers: {}, doNotIndent: {"!cdata": true}};
var UseKludges = Kludges;
var alignCDATA = false;
// Simple stateful tokenizer for XML documents. Returns a
// MochiKit-style iterator, with a state property that contains a
// function encapsulating the current state. See tokenize.js.
var tokenizeXML = (function() {
function inText(source, setState) {
var ch = source.next();
if (ch == "<") {
if (source.equals("!")) {
source.next();
if (source.equals("[")) {
if (source.lookAhead("[CDATA[", true)) {
setState(inBlock("xml-cdata", "]]>"));
return null;
}
else {
return "xml-text";
}
}
else if (source.lookAhead("--", true)) {
setState(inBlock("xml-comment", "-->"));
return null;
}
else {
return "xml-text";
}
}
else if (source.equals("?")) {
source.next();
source.nextWhileMatches(/[\w\._\-]/);
setState(inBlock("xml-processing", "?>"));
return "xml-processing";
}
else {
if (source.equals("/")) source.next();
setState(inTag);
return "xml-punctuation";
}
}
else if (ch == "&") {
while (!source.endOfLine()) {
if (source.next() == ";")
break;
}
return "xml-entity";
}
else {
source.nextWhileMatches(/[^&<\n]/);
return "xml-text";
}
}
function inTag(source, setState) {
var ch = source.next();
if (ch == ">") {
setState(inText);
return "xml-punctuation";
}
else if (/[?\/]/.test(ch) && source.equals(">")) {
source.next();
setState(inText);
return "xml-punctuation";
}
else if (ch == "=") {
return "xml-punctuation";
}
else if (/[\'\"]/.test(ch)) {
setState(inAttribute(ch));
return null;
}
else {
source.nextWhileMatches(/[^\s\u00a0=<>\"\'\/?]/);
return "xml-name";
}
}
function inAttribute(quote) {
return function(source, setState) {
while (!source.endOfLine()) {
if (source.next() == quote) {
setState(inTag);
break;
}
}
return "xml-attribute";
};
}
function inBlock(style, terminator) {
return function(source, setState) {
while (!source.endOfLine()) {
if (source.lookAhead(terminator, true)) {
setState(inText);
break;
}
source.next();
}
return style;
};
}
return function(source, startState) {
return tokenizer(source, startState || inText);
};
})();
// The parser. The structure of this function largely follows that of
// parseJavaScript in parsejavascript.js (there is actually a bit more
// shared code than I'd like), but it is quite a bit simpler.
function parseXML(source) {
var tokens = tokenizeXML(source);
var cc = [base];
var tokenNr = 0, indented = 0;
var currentTag = null, context = null;
var consume, marked;
function push(fs) {
for (var i = fs.length - 1; i >= 0; i--)
cc.push(fs[i]);
}
function cont() {
push(arguments);
consume = true;
}
function pass() {
push(arguments);
consume = false;
}
function mark(style) {
marked = style;
}
function expect(text) {
return function(style, content) {
if (content == text) cont();
else mark("xml-error") || cont(arguments.callee);
};
}
function pushContext(tagname, startOfLine) {
var noIndent = UseKludges.doNotIndent.hasOwnProperty(tagname) || (context && context.noIndent);
context = {prev: context, name: tagname, indent: indented, startOfLine: startOfLine, noIndent: noIndent};
}
function popContext() {
context = context.prev;
}
function computeIndentation(baseContext) {
return function(nextChars, current) {
var context = baseContext;
if (context && context.noIndent)
return current;
if (alignCDATA && /<!\[CDATA\[/.test(nextChars))
return 0;
if (context && /^<\//.test(nextChars))
context = context.prev;
while (context && !context.startOfLine)
context = context.prev;
if (context)
return context.indent + indentUnit;
else
return 0;
};
}
function base() {
return pass(element, base);
}
var harmlessTokens = {"xml-text": true, "xml-entity": true, "xml-comment": true, "xml-processing": true};
function element(style, content) {
if (content == "<") cont(tagname, attributes, endtag(tokenNr == 1));
else if (content == "</") cont(closetagname, expect(">"));
else if (style == "xml-cdata") {
if (!context || context.name != "!cdata") pushContext("!cdata");
if (/\]\]>$/.test(content)) popContext();
cont();
}
else if (harmlessTokens.hasOwnProperty(style)) cont();
else mark("xml-error") || cont();
}
function tagname(style, content) {
if (style == "xml-name") {
currentTag = content.toLowerCase();
mark("xml-tagname");
cont();
}
else {
currentTag = null;
pass();
}
}
function closetagname(style, content) {
if (style == "xml-name" && context && content.toLowerCase() == context.name) {
popContext();
mark("xml-tagname");
}
else {
mark("xml-error");
}
cont();
}
function endtag(startOfLine) {
return function(style, content) {
if (content == "/>" || (content == ">" && UseKludges.autoSelfClosers.hasOwnProperty(currentTag))) cont();
else if (content == ">") pushContext(currentTag, startOfLine) || cont();
else mark("xml-error") || cont(arguments.callee);
};
}
function attributes(style) {
if (style == "xml-name") mark("xml-attname") || cont(attribute, attributes);
else pass();
}
function attribute(style, content) {
if (content == "=") cont(value);
else if (content == ">" || content == "/>") pass(endtag);
else pass();
}
function value(style) {
if (style == "xml-attribute") cont(value);
else pass();
}
return {
indentation: function() {return indented;},
next: function(){
var token = tokens.next();
if (token.style == "whitespace" && tokenNr == 0)
indented = token.value.length;
else
tokenNr++;
if (token.content == "\n") {
indented = tokenNr = 0;
token.indentation = computeIndentation(context);
}
if (token.style == "whitespace" || token.type == "xml-comment")
return token;
while(true){
consume = marked = false;
cc.pop()(token.style, token.content);
if (consume){
if (marked)
token.style = marked;
return token;
}
}
},
copy: function(){
var _cc = cc.concat([]), _tokenState = tokens.state, _context = context;
var parser = this;
return function(input){
cc = _cc.concat([]);
tokenNr = indented = 0;
context = _context;
tokens = tokenizeXML(input, _tokenState);
return parser;
};
}
};
}
return {
make: parseXML,
electricChars: "/",
configure: function(config) {
if (config.useHTMLKludges != null)
UseKludges = config.useHTMLKludges ? Kludges : NoKludges;
if (config.alignCDATA)
alignCDATA = config.alignCDATA;
}
};
})();

View File

@ -0,0 +1,583 @@
/* Functionality for finding, storing, and restoring selections
*
* This does not provide a generic API, just the minimal functionality
* required by the CodeMirror system.
*/
// Namespace object.
var select = {};
(function() {
select.ie_selection = document.selection && document.selection.createRangeCollection;
// Find the 'top-level' (defined as 'a direct child of the node
// passed as the top argument') node that the given node is
// contained in. Return null if the given node is not inside the top
// node.
function topLevelNodeAt(node, top) {
while (node && node.parentNode != top)
node = node.parentNode;
return node;
}
// Find the top-level node that contains the node before this one.
function topLevelNodeBefore(node, top) {
while (!node.previousSibling && node.parentNode != top)
node = node.parentNode;
return topLevelNodeAt(node.previousSibling, top);
}
// Used to prevent restoring a selection when we do not need to.
var currentSelection = null;
var fourSpaces = "\u00a0\u00a0\u00a0\u00a0";
select.snapshotChanged = function() {
if (currentSelection) currentSelection.changed = true;
};
// This is called by the code in editor.js whenever it is replacing
// a text node. The function sees whether the given oldNode is part
// of the current selection, and updates this selection if it is.
// Because nodes are often only partially replaced, the length of
// the part that gets replaced has to be taken into account -- the
// selection might stay in the oldNode if the newNode is smaller
// than the selection's offset. The offset argument is needed in
// case the selection does move to the new object, and the given
// length is not the whole length of the new node (part of it might
// have been used to replace another node).
select.snapshotReplaceNode = function(from, to, length, offset) {
if (!currentSelection) return;
currentSelection.changed = true;
function replace(point) {
if (from == point.node) {
if (length && point.offset > length) {
point.offset -= length;
}
else {
point.node = to;
point.offset += (offset || 0);
}
}
}
replace(currentSelection.start);
replace(currentSelection.end);
};
select.snapshotMove = function(from, to, distance, relative, ifAtStart) {
if (!currentSelection) return;
currentSelection.changed = true;
function move(point) {
if (from == point.node && (!ifAtStart || point.offset == 0)) {
point.node = to;
if (relative) point.offset = Math.max(0, point.offset + distance);
else point.offset = distance;
}
}
move(currentSelection.start);
move(currentSelection.end);
};
// Most functions are defined in two ways, one for the IE selection
// model, one for the W3C one.
if (select.ie_selection) {
function selectionNode(win, start) {
var range = win.document.selection.createRange();
range.collapse(start);
function nodeAfter(node) {
var found = null;
while (!found && node) {
found = node.nextSibling;
node = node.parentNode;
}
return nodeAtStartOf(found);
}
function nodeAtStartOf(node) {
while (node && node.firstChild) node = node.firstChild;
return {node: node, offset: 0};
}
var containing = range.parentElement();
if (!isAncestor(win.document.body, containing)) return null;
if (!containing.firstChild) return nodeAtStartOf(containing);
var working = range.duplicate();
working.moveToElementText(containing);
working.collapse(true);
for (var cur = containing.firstChild; cur; cur = cur.nextSibling) {
if (cur.nodeType == 3) {
var size = cur.nodeValue.length;
working.move("character", size);
}
else {
working.moveToElementText(cur);
working.collapse(false);
}
var dir = range.compareEndPoints("StartToStart", working);
if (dir == 0) return nodeAfter(cur);
if (dir == 1) continue;
if (cur.nodeType != 3) return nodeAtStartOf(cur);
working.setEndPoint("StartToEnd", range);
return {node: cur, offset: size - working.text.length};
}
return nodeAfter(containing);
}
select.markSelection = function(win) {
currentSelection = null;
var sel = win.document.selection;
if (!sel) return;
var start = selectionNode(win, true),
end = selectionNode(win, false);
if (!start || !end) return;
currentSelection = {start: start, end: end, window: win, changed: false};
};
select.selectMarked = function() {
if (!currentSelection || !currentSelection.changed) return;
function makeRange(point) {
var range = currentSelection.window.document.body.createTextRange();
var node = point.node;
if (!node) {
range.moveToElementText(currentSelection.window.document.body);
range.collapse(false);
}
else if (node.nodeType == 3) {
range.moveToElementText(node.parentNode);
var offset = point.offset;
while (node.previousSibling) {
node = node.previousSibling;
offset += (node.innerText || "").length;
}
range.move("character", offset);
}
else {
range.moveToElementText(node);
range.collapse(true);
}
return range;
}
var start = makeRange(currentSelection.start), end = makeRange(currentSelection.end);
start.setEndPoint("StartToEnd", end);
start.select();
};
// Get the top-level node that one end of the cursor is inside or
// after. Note that this returns false for 'no cursor', and null
// for 'start of document'.
select.selectionTopNode = function(container, start) {
var selection = container.ownerDocument.selection;
if (!selection) return false;
var range = selection.createRange();
range.collapse(start);
var around = range.parentElement();
if (around && isAncestor(container, around)) {
// Only use this node if the selection is not at its start.
var range2 = range.duplicate();
range2.moveToElementText(around);
if (range.compareEndPoints("StartToStart", range2) == -1)
return topLevelNodeAt(around, container);
}
// Fall-back hack
try {range.pasteHTML("<span id='xxx-temp-xxx'></span>");}
catch (e) {return false;}
var temp = container.ownerDocument.getElementById("xxx-temp-xxx");
if (temp) {
var result = topLevelNodeBefore(temp, container);
removeElement(temp);
return result;
}
return false;
};
// Place the cursor after this.start. This is only useful when
// manually moving the cursor instead of restoring it to its old
// position.
select.focusAfterNode = function(node, container) {
var range = container.ownerDocument.body.createTextRange();
range.moveToElementText(node || container);
range.collapse(!node);
range.select();
};
select.somethingSelected = function(win) {
var sel = win.document.selection;
return sel && (sel.createRange().text != "");
};
function insertAtCursor(window, html) {
var selection = window.document.selection;
if (selection) {
var range = selection.createRange();
range.pasteHTML(html);
range.collapse(false);
range.select();
}
}
// Used to normalize the effect of the enter key, since browsers
// do widely different things when pressing enter in designMode.
select.insertNewlineAtCursor = function(window) {
insertAtCursor(window, "<br>");
};
select.insertTabAtCursor = function(window) {
insertAtCursor(window, fourSpaces);
};
// Get the BR node at the start of the line on which the cursor
// currently is, and the offset into the line. Returns null as
// node if cursor is on first line.
select.cursorPos = function(container, start) {
var selection = container.ownerDocument.selection;
if (!selection) return null;
var topNode = select.selectionTopNode(container, start);
while (topNode && topNode.nodeName != "BR")
topNode = topNode.previousSibling;
var range = selection.createRange(), range2 = range.duplicate();
range.collapse(start);
if (topNode) {
range2.moveToElementText(topNode);
range2.collapse(false);
}
else {
// When nothing is selected, we can get all kinds of funky errors here.
try { range2.moveToElementText(container); }
catch (e) { return null; }
range2.collapse(true);
}
range.setEndPoint("StartToStart", range2);
return {node: topNode, offset: range.text.length};
};
select.setCursorPos = function(container, from, to) {
function rangeAt(pos) {
var range = container.ownerDocument.body.createTextRange();
if (!pos.node) {
range.moveToElementText(container);
range.collapse(true);
}
else {
range.moveToElementText(pos.node);
range.collapse(false);
}
range.move("character", pos.offset);
return range;
}
var range = rangeAt(from);
if (to && to != from)
range.setEndPoint("EndToEnd", rangeAt(to));
range.select();
}
// Make sure the cursor is visible.
select.scrollToCursor = function(container) {
var selection = container.ownerDocument.selection;
if (!selection) return null;
selection.createRange().scrollIntoView();
};
select.scrollToNode = function(node) {
if (!node) return;
node.scrollIntoView();
};
// Some hacks for storing and re-storing the selection when the editor loses and regains focus.
select.selectionCoords = function (win) {
var selection = win.document.selection;
if (!selection) return null;
var start = selection.createRange(), end = start.duplicate();
start.collapse(true);
end.collapse(false);
var body = win.document.body;
return {start: {x: start.boundingLeft + body.scrollLeft - 1,
y: start.boundingTop + body.scrollTop},
end: {x: end.boundingLeft + body.scrollLeft - 1,
y: end.boundingTop + body.scrollTop}};
};
// Restore a stored selection.
select.selectCoords = function(win, coords) {
if (!coords) return;
var range1 = win.document.body.createTextRange(), range2 = range1.duplicate();
// This can fail for various hard-to-handle reasons.
try {
range1.moveToPoint(coords.start.x, coords.start.y);
range2.moveToPoint(coords.end.x, coords.end.y);
range1.setEndPoint("EndToStart", range2);
range1.select();
} catch(e) {alert(e.message);}
};
}
// W3C model
else {
// Store start and end nodes, and offsets within these, and refer
// back to the selection object from those nodes, so that this
// object can be updated when the nodes are replaced before the
// selection is restored.
select.markSelection = function (win) {
var selection = win.getSelection();
if (!selection || selection.rangeCount == 0)
return (currentSelection = null);
var range = selection.getRangeAt(0);
currentSelection = {
start: {node: range.startContainer, offset: range.startOffset},
end: {node: range.endContainer, offset: range.endOffset},
window: win,
changed: false
};
// We want the nodes right at the cursor, not one of their
// ancestors with a suitable offset. This goes down the DOM tree
// until a 'leaf' is reached (or is it *up* the DOM tree?).
function normalize(point){
while (point.node.nodeType != 3 && point.node.nodeName != "BR") {
var newNode = point.node.childNodes[point.offset] || point.node.nextSibling;
point.offset = 0;
while (!newNode && point.node.parentNode) {
point.node = point.node.parentNode;
newNode = point.node.nextSibling;
}
point.node = newNode;
if (!newNode)
break;
}
}
normalize(currentSelection.start);
normalize(currentSelection.end);
};
select.selectMarked = function () {
if (!currentSelection || !currentSelection.changed) return;
var win = currentSelection.window, range = win.document.createRange();
function setPoint(point, which) {
if (point.node) {
// Some magic to generalize the setting of the start and end
// of a range.
if (point.offset == 0)
range["set" + which + "Before"](point.node);
else
range["set" + which](point.node, point.offset);
}
else {
range.setStartAfter(win.document.body.lastChild || win.document.body);
}
}
setPoint(currentSelection.end, "End");
setPoint(currentSelection.start, "Start");
selectRange(range, win);
};
// Helper for selecting a range object.
function selectRange(range, window) {
var selection = window.getSelection();
selection.removeAllRanges();
selection.addRange(range);
};
function selectionRange(window) {
var selection = window.getSelection();
if (!selection || selection.rangeCount == 0)
return false;
else
return selection.getRangeAt(0);
}
// Finding the top-level node at the cursor in the W3C is, as you
// can see, quite an involved process.
select.selectionTopNode = function(container, start) {
var range = selectionRange(container.ownerDocument.defaultView);
if (!range) return false;
var node = start ? range.startContainer : range.endContainer;
var offset = start ? range.startOffset : range.endOffset;
// Work around (yet another) bug in Opera's selection model.
if (window.opera && !start && range.endContainer == container && range.endOffset == range.startOffset + 1 &&
container.childNodes[range.startOffset] && container.childNodes[range.startOffset].nodeName == "BR")
offset--;
// For text nodes, we look at the node itself if the cursor is
// inside, or at the node before it if the cursor is at the
// start.
if (node.nodeType == 3){
if (offset > 0)
return topLevelNodeAt(node, container);
else
return topLevelNodeBefore(node, container);
}
// Occasionally, browsers will return the HTML node as
// selection. If the offset is 0, we take the start of the frame
// ('after null'), otherwise, we take the last node.
else if (node.nodeName == "HTML") {
return (offset == 1 ? null : container.lastChild);
}
// If the given node is our 'container', we just look up the
// correct node by using the offset.
else if (node == container) {
return (offset == 0) ? null : node.childNodes[offset - 1];
}
// In any other case, we have a regular node. If the cursor is
// at the end of the node, we use the node itself, if it is at
// the start, we use the node before it, and in any other
// case, we look up the child before the cursor and use that.
else {
if (offset == node.childNodes.length)
return topLevelNodeAt(node, container);
else if (offset == 0)
return topLevelNodeBefore(node, container);
else
return topLevelNodeAt(node.childNodes[offset - 1], container);
}
};
select.focusAfterNode = function(node, container) {
var win = container.ownerDocument.defaultView,
range = win.document.createRange();
range.setStartBefore(container.firstChild || container);
// In Opera, setting the end of a range at the end of a line
// (before a BR) will cause the cursor to appear on the next
// line, so we set the end inside of the start node when
// possible.
if (node && !node.firstChild)
range.setEndAfter(node);
else if (node)
range.setEnd(node, node.childNodes.length);
else
range.setEndBefore(container.firstChild || container);
range.collapse(false);
selectRange(range, win);
};
select.somethingSelected = function(win) {
var range = selectionRange(win);
return range && !range.collapsed;
};
function insertNodeAtCursor(window, node) {
var range = selectionRange(window);
if (!range) return;
range.deleteContents();
range.insertNode(node);
webkitLastLineHack(window.document.body);
range = window.document.createRange();
range.selectNode(node);
range.collapse(false);
selectRange(range, window);
}
select.insertNewlineAtCursor = function(window) {
insertNodeAtCursor(window, window.document.createElement("BR"));
};
select.insertTabAtCursor = function(window) {
insertNodeAtCursor(window, window.document.createTextNode(fourSpaces));
};
select.cursorPos = function(container, start) {
var range = selectionRange(window);
if (!range) return;
var topNode = select.selectionTopNode(container, start);
while (topNode && topNode.nodeName != "BR")
topNode = topNode.previousSibling;
range = range.cloneRange();
range.collapse(start);
if (topNode)
range.setStartAfter(topNode);
else
range.setStartBefore(container);
return {node: topNode, offset: range.toString().length};
};
select.setCursorPos = function(container, from, to) {
var win = container.ownerDocument.defaultView,
range = win.document.createRange();
function setPoint(node, offset, side) {
if (!node)
node = container.firstChild;
else
node = node.nextSibling;
if (!node)
return;
if (offset == 0) {
range["set" + side + "Before"](node);
return true;
}
var backlog = []
function decompose(node) {
if (node.nodeType == 3)
backlog.push(node);
else
forEach(node.childNodes, decompose);
}
while (true) {
while (node && !backlog.length) {
decompose(node);
node = node.nextSibling;
}
var cur = backlog.shift();
if (!cur) return false;
var length = cur.nodeValue.length;
if (length >= offset) {
range["set" + side](cur, offset);
return true;
}
offset -= length;
}
}
to = to || from;
if (setPoint(to.node, to.offset, "End") && setPoint(from.node, from.offset, "Start"))
selectRange(range, win);
};
select.scrollToNode = function(element) {
if (!element) return;
var doc = element.ownerDocument, body = doc.body, win = doc.defaultView, html = doc.documentElement;
// In Opera, BR elements *always* have a scrollTop property of zero. Go Opera.
while (element && !element.offsetTop)
element = element.previousSibling;
var y = 0, pos = element;
while (pos && pos.offsetParent) {
y += pos.offsetTop;
pos = pos.offsetParent;
}
var screen_y = y - (body.scrollTop || html.scrollTop || 0);
if (screen_y < 0 || screen_y > win.innerHeight - 30)
win.scrollTo(body.scrollLeft || html.scrollLeft || 0, y);
};
select.scrollToCursor = function(container) {
select.scrollToNode(select.selectionTopNode(container, true) || container.firstChild);
};
}
})();

View File

@ -0,0 +1,140 @@
/* String streams are the things fed to parsers (which can feed them
* to a tokenizer if they want). They provide peek and next methods
* for looking at the current character (next 'consumes' this
* character, peek does not), and a get method for retrieving all the
* text that was consumed since the last time get was called.
*
* An easy mistake to make is to let a StopIteration exception finish
* the token stream while there are still characters pending in the
* string stream (hitting the end of the buffer while parsing a
* token). To make it easier to detect such errors, the strings throw
* an exception when this happens.
*/
// Make a string stream out of an iterator that returns strings. This
// is applied to the result of traverseDOM (see codemirror.js), and
// the resulting stream is fed to the parser.
window.stringStream = function(source){
// String that's currently being iterated over.
var current = "";
// Position in that string.
var pos = 0;
// Accumulator for strings that have been iterated over but not
// get()-ed yet.
var accum = "";
// Make sure there are more characters ready, or throw
// StopIteration.
function ensureChars() {
while (pos == current.length) {
accum += current;
current = ""; // In case source.next() throws
pos = 0;
try {current = source.next();}
catch (e) {
if (e != StopIteration) throw e;
else return false;
}
}
return true;
}
return {
// Return the next character in the stream.
peek: function() {
if (!ensureChars()) return null;
return current.charAt(pos);
},
// Get the next character, throw StopIteration if at end, check
// for unused content.
next: function() {
if (!ensureChars()) {
if (accum.length > 0)
throw "End of stringstream reached without emptying buffer ('" + accum + "').";
else
throw StopIteration;
}
return current.charAt(pos++);
},
// Return the characters iterated over since the last call to
// .get().
get: function() {
var temp = accum;
accum = "";
if (pos > 0){
temp += current.slice(0, pos);
current = current.slice(pos);
pos = 0;
}
return temp;
},
// Push a string back into the stream.
push: function(str) {
current = current.slice(0, pos) + str + current.slice(pos);
},
lookAhead: function(str, consume, skipSpaces, caseInsensitive) {
function cased(str) {return caseInsensitive ? str.toLowerCase() : str;}
str = cased(str);
var found = false;
var _accum = accum, _pos = pos;
if (skipSpaces) this.nextWhileMatches(/[\s\u00a0]/);
while (true) {
var end = pos + str.length, left = current.length - pos;
if (end <= current.length) {
found = str == cased(current.slice(pos, end));
pos = end;
break;
}
else if (str.slice(0, left) == cased(current.slice(pos))) {
accum += current; current = "";
try {current = source.next();}
catch (e) {break;}
pos = 0;
str = str.slice(left);
}
else {
break;
}
}
if (!(found && consume)) {
current = accum.slice(_accum.length) + current;
pos = _pos;
accum = _accum;
}
return found;
},
// Utils built on top of the above
more: function() {
return this.peek() !== null;
},
applies: function(test) {
var next = this.peek();
return (next !== null && test(next));
},
nextWhile: function(test) {
var next;
while ((next = this.peek()) !== null && test(next))
this.next();
},
matches: function(re) {
var next = this.peek();
return (next !== null && re.test(next));
},
nextWhileMatches: function(re) {
var next;
while ((next = this.peek()) !== null && re.test(next))
this.next();
},
equals: function(ch) {
return ch === this.peek();
},
endOfLine: function() {
var next = this.peek();
return next == null || next == "\n";
}
};
};

View File

@ -0,0 +1,57 @@
// A framework for simple tokenizers. Takes care of newlines and
// white-space, and of getting the text from the source stream into
// the token object. A state is a function of two arguments -- a
// string stream and a setState function. The second can be used to
// change the tokenizer's state, and can be ignored for stateless
// tokenizers. This function should advance the stream over a token
// and return a string or object containing information about the next
// token, or null to pass and have the (new) state be called to finish
// the token. When a string is given, it is wrapped in a {style, type}
// object. In the resulting object, the characters consumed are stored
// under the content property. Any whitespace following them is also
// automatically consumed, and added to the value property. (Thus,
// content is the actual meaningful part of the token, while value
// contains all the text it spans.)
function tokenizer(source, state) {
// Newlines are always a separate token.
function isWhiteSpace(ch) {
// The messy regexp is because IE's regexp matcher is of the
// opinion that non-breaking spaces are no whitespace.
return ch != "\n" && /^[\s\u00a0]*$/.test(ch);
}
var tokenizer = {
state: state,
take: function(type) {
if (typeof(type) == "string")
type = {style: type, type: type};
type.content = (type.content || "") + source.get();
if (!/\n$/.test(type.content))
source.nextWhile(isWhiteSpace);
type.value = type.content + source.get();
return type;
},
next: function () {
if (!source.more()) throw StopIteration;
var type;
if (source.equals("\n")) {
source.next();
return this.take("whitespace");
}
if (source.applies(isWhiteSpace))
type = "whitespace";
else
while (!type)
type = this.state(source, function(s) {tokenizer.state = s;});
return this.take(type);
}
};
return tokenizer;
}

View File

@ -0,0 +1,175 @@
/* Tokenizer for JavaScript code */
var tokenizeJavaScript = (function() {
// Advance the stream until the given character (not preceded by a
// backslash) is encountered, or the end of the line is reached.
function nextUntilUnescaped(source, end) {
var escaped = false;
var next;
while (!source.endOfLine()) {
var next = source.next();
if (next == end && !escaped)
return false;
escaped = !escaped && next == "\\";
}
return escaped;
}
// A map of JavaScript's keywords. The a/b/c keyword distinction is
// very rough, but it gives the parser enough information to parse
// correct code correctly (we don't care that much how we parse
// incorrect code). The style information included in these objects
// is used by the highlighter to pick the correct CSS style for a
// token.
var keywords = function(){
function result(type, style){
return {type: type, style: "js-" + style};
}
// keywords that take a parenthised expression, and then a
// statement (if)
var keywordA = result("keyword a", "keyword");
// keywords that take just a statement (else)
var keywordB = result("keyword b", "keyword");
// keywords that optionally take an expression, and form a
// statement (return)
var keywordC = result("keyword c", "keyword");
var operator = result("operator", "keyword");
var atom = result("atom", "atom");
return {
"if": keywordA, "while": keywordA, "with": keywordA,
"else": keywordB, "do": keywordB, "try": keywordB, "finally": keywordB,
"return": keywordC, "break": keywordC, "continue": keywordC, "new": keywordC, "delete": keywordC, "throw": keywordC,
"in": operator, "typeof": operator, "instanceof": operator,
"var": result("var", "keyword"), "function": result("function", "keyword"), "catch": result("catch", "keyword"),
"for": result("for", "keyword"), "switch": result("switch", "keyword"),
"case": result("case", "keyword"), "default": result("default", "keyword"),
"true": atom, "false": atom, "null": atom, "undefined": atom, "NaN": atom, "Infinity": atom
};
}();
// Some helper regexps
var isOperatorChar = /[+\-*&%\/=<>!?|]/;
var isHexDigit = /[0-9A-Fa-f]/;
var isWordChar = /[\w\$_]/;
// Wrapper around jsToken that helps maintain parser state (whether
// we are inside of a multi-line comment and whether the next token
// could be a regular expression).
function jsTokenState(inside, regexp) {
return function(source, setState) {
var newInside = inside;
var type = jsToken(inside, regexp, source, function(c) {newInside = c;});
var newRegexp = type.type == "operator" || type.type == "keyword c" || type.type.match(/^[\[{}\(,;:]$/);
if (newRegexp != regexp || newInside != inside)
setState(jsTokenState(newInside, newRegexp));
return type;
};
}
// The token reader, inteded to be used by the tokenizer from
// tokenize.js (through jsTokenState). Advances the source stream
// over a token, and returns an object containing the type and style
// of that token.
function jsToken(inside, regexp, source, setInside) {
function readHexNumber(){
source.next(); // skip the 'x'
source.nextWhileMatches(isHexDigit);
return {type: "number", style: "js-atom"};
}
function readNumber() {
source.nextWhileMatches(/[0-9]/);
if (source.equals(".")){
source.next();
source.nextWhileMatches(/[0-9]/);
}
if (source.equals("e") || source.equals("E")){
source.next();
if (source.equals("-"))
source.next();
source.nextWhileMatches(/[0-9]/);
}
return {type: "number", style: "js-atom"};
}
// Read a word, look it up in keywords. If not found, it is a
// variable, otherwise it is a keyword of the type found.
function readWord() {
source.nextWhileMatches(isWordChar);
var word = source.get();
var known = keywords.hasOwnProperty(word) && keywords.propertyIsEnumerable(word) && keywords[word];
return known ? {type: known.type, style: known.style, content: word} :
{type: "variable", style: "js-variable", content: word};
}
function readRegexp() {
nextUntilUnescaped(source, "/");
source.nextWhileMatches(/[gi]/);
return {type: "regexp", style: "js-string"};
}
// Mutli-line comments are tricky. We want to return the newlines
// embedded in them as regular newline tokens, and then continue
// returning a comment token for every line of the comment. So
// some state has to be saved (inside) to indicate whether we are
// inside a /* */ sequence.
function readMultilineComment(start){
var newInside = "/*";
var maybeEnd = (start == "*");
while (true) {
if (source.endOfLine())
break;
var next = source.next();
if (next == "/" && maybeEnd){
newInside = null;
break;
}
maybeEnd = (next == "*");
}
setInside(newInside);
return {type: "comment", style: "js-comment"};
}
function readOperator() {
source.nextWhileMatches(isOperatorChar);
return {type: "operator", style: "js-operator"};
}
function readString(quote) {
var endBackSlash = nextUntilUnescaped(source, quote);
setInside(endBackSlash ? quote : null);
return {type: "string", style: "js-string"};
}
// Fetch the next token. Dispatches on first character in the
// stream, or first two characters when the first is a slash.
if (inside == "\"" || inside == "'")
return readString(inside);
var ch = source.next();
if (inside == "/*")
return readMultilineComment(ch);
else if (ch == "\"" || ch == "'")
return readString(ch);
// with punctuation, the type of the token is the symbol itself
else if (/[\[\]{}\(\),;\:\.]/.test(ch))
return {type: ch, style: "js-punctuation"};
else if (ch == "0" && (source.equals("x") || source.equals("X")))
return readHexNumber();
else if (/[0-9]/.test(ch))
return readNumber();
else if (ch == "/"){
if (source.equals("*"))
{ source.next(); return readMultilineComment(ch); }
else if (source.equals("/"))
{ nextUntilUnescaped(source, null); return {type: "comment", style: "js-comment"};}
else if (regexp)
return readRegexp();
else
return readOperator();
}
else if (isOperatorChar.test(ch))
return readOperator();
else
return readWord();
}
// The external interface to the tokenizer.
return function(source, startState) {
return tokenizer(source, startState || jsTokenState(false, true));
};
})();

View File

@ -0,0 +1,403 @@
/**
* Storage and control for undo information within a CodeMirror
* editor. 'Why on earth is such a complicated mess required for
* that?', I hear you ask. The goal, in implementing this, was to make
* the complexity of storing and reverting undo information depend
* only on the size of the edited or restored content, not on the size
* of the whole document. This makes it necessary to use a kind of
* 'diff' system, which, when applied to a DOM tree, causes some
* complexity and hackery.
*
* In short, the editor 'touches' BR elements as it parses them, and
* the History stores these. When nothing is touched in commitDelay
* milliseconds, the changes are committed: It goes over all touched
* nodes, throws out the ones that did not change since last commit or
* are no longer in the document, and assembles the rest into zero or
* more 'chains' -- arrays of adjacent lines. Links back to these
* chains are added to the BR nodes, while the chain that previously
* spanned these nodes is added to the undo history. Undoing a change
* means taking such a chain off the undo history, restoring its
* content (text is saved per line) and linking it back into the
* document.
*/
// A history object needs to know about the DOM container holding the
// document, the maximum amount of undo levels it should store, the
// delay (of no input) after which it commits a set of changes, and,
// unfortunately, the 'parent' window -- a window that is not in
// designMode, and on which setTimeout works in every browser.
function History(container, maxDepth, commitDelay, editor, onChange) {
this.container = container;
this.maxDepth = maxDepth; this.commitDelay = commitDelay;
this.editor = editor; this.parent = editor.parent;
this.onChange = onChange;
// This line object represents the initial, empty editor.
var initial = {text: "", from: null, to: null};
// As the borders between lines are represented by BR elements, the
// start of the first line and the end of the last one are
// represented by null. Since you can not store any properties
// (links to line objects) in null, these properties are used in
// those cases.
this.first = initial; this.last = initial;
// Similarly, a 'historyTouched' property is added to the BR in
// front of lines that have already been touched, and 'firstTouched'
// is used for the first line.
this.firstTouched = false;
// History is the set of committed changes, touched is the set of
// nodes touched since the last commit.
this.history = []; this.redoHistory = []; this.touched = [];
}
History.prototype = {
// Schedule a commit (if no other touches come in for commitDelay
// milliseconds).
scheduleCommit: function() {
var self = this;
this.parent.clearTimeout(this.commitTimeout);
this.commitTimeout = this.parent.setTimeout(function(){self.tryCommit();}, this.commitDelay);
},
// Mark a node as touched. Null is a valid argument.
touch: function(node) {
this.setTouched(node);
this.scheduleCommit();
},
// Undo the last change.
undo: function() {
// Make sure pending changes have been committed.
this.commit();
if (this.history.length) {
// Take the top diff from the history, apply it, and store its
// shadow in the redo history.
var item = this.history.pop();
this.redoHistory.push(this.updateTo(item, "applyChain"));
if (this.onChange) this.onChange();
return this.chainNode(item);
}
},
// Redo the last undone change.
redo: function() {
this.commit();
if (this.redoHistory.length) {
// The inverse of undo, basically.
var item = this.redoHistory.pop();
this.addUndoLevel(this.updateTo(item, "applyChain"));
if (this.onChange) this.onChange();
return this.chainNode(item);
}
},
clear: function() {
this.history = [];
this.redoHistory = [];
},
// Ask for the size of the un/redo histories.
historySize: function() {
return {undo: this.history.length, redo: this.redoHistory.length};
},
// Push a changeset into the document.
push: function(from, to, lines) {
var chain = [];
for (var i = 0; i < lines.length; i++) {
var end = (i == lines.length - 1) ? to : this.container.ownerDocument.createElement("BR");
chain.push({from: from, to: end, text: cleanText(lines[i])});
from = end;
}
this.pushChains([chain], from == null && to == null);
},
pushChains: function(chains, doNotHighlight) {
this.commit(doNotHighlight);
this.addUndoLevel(this.updateTo(chains, "applyChain"));
this.redoHistory = [];
},
// Retrieve a DOM node from a chain (for scrolling to it after undo/redo).
chainNode: function(chains) {
for (var i = 0; i < chains.length; i++) {
var start = chains[i][0], node = start && (start.from || start.to);
if (node) return node;
}
},
// Clear the undo history, make the current document the start
// position.
reset: function() {
this.history = []; this.redoHistory = [];
},
textAfter: function(br) {
return this.after(br).text;
},
nodeAfter: function(br) {
return this.after(br).to;
},
nodeBefore: function(br) {
return this.before(br).from;
},
// Commit unless there are pending dirty nodes.
tryCommit: function() {
if (!window.History) return; // Stop when frame has been unloaded
if (this.editor.highlightDirty()) this.commit();
else this.scheduleCommit();
},
// Check whether the touched nodes hold any changes, if so, commit
// them.
commit: function(doNotHighlight) {
this.parent.clearTimeout(this.commitTimeout);
// Make sure there are no pending dirty nodes.
if (!doNotHighlight) this.editor.highlightDirty(true);
// Build set of chains.
var chains = this.touchedChains(), self = this;
if (chains.length) {
this.addUndoLevel(this.updateTo(chains, "linkChain"));
this.redoHistory = [];
if (this.onChange) this.onChange();
}
},
// [ end of public interface ]
// Update the document with a given set of chains, return its
// shadow. updateFunc should be "applyChain" or "linkChain". In the
// second case, the chains are taken to correspond the the current
// document, and only the state of the line data is updated. In the
// first case, the content of the chains is also pushed iinto the
// document.
updateTo: function(chains, updateFunc) {
var shadows = [], dirty = [];
for (var i = 0; i < chains.length; i++) {
shadows.push(this.shadowChain(chains[i]));
dirty.push(this[updateFunc](chains[i]));
}
if (updateFunc == "applyChain")
this.notifyDirty(dirty);
return shadows;
},
// Notify the editor that some nodes have changed.
notifyDirty: function(nodes) {
forEach(nodes, method(this.editor, "addDirtyNode"))
this.editor.scheduleHighlight();
},
// Link a chain into the DOM nodes (or the first/last links for null
// nodes).
linkChain: function(chain) {
for (var i = 0; i < chain.length; i++) {
var line = chain[i];
if (line.from) line.from.historyAfter = line;
else this.first = line;
if (line.to) line.to.historyBefore = line;
else this.last = line;
}
},
// Get the line object after/before a given node.
after: function(node) {
return node ? node.historyAfter : this.first;
},
before: function(node) {
return node ? node.historyBefore : this.last;
},
// Mark a node as touched if it has not already been marked.
setTouched: function(node) {
if (node) {
if (!node.historyTouched) {
this.touched.push(node);
node.historyTouched = true;
}
}
else {
this.firstTouched = true;
}
},
// Store a new set of undo info, throw away info if there is more of
// it than allowed.
addUndoLevel: function(diffs) {
this.history.push(diffs);
if (this.history.length > this.maxDepth)
this.history.shift();
},
// Build chains from a set of touched nodes.
touchedChains: function() {
var self = this;
// The temp system is a crummy hack to speed up determining
// whether a (currently touched) node has a line object associated
// with it. nullTemp is used to store the object for the first
// line, other nodes get it stored in their historyTemp property.
var nullTemp = null;
function temp(node) {return node ? node.historyTemp : nullTemp;}
function setTemp(node, line) {
if (node) node.historyTemp = line;
else nullTemp = line;
}
function buildLine(node) {
var text = [];
for (var cur = node ? node.nextSibling : self.container.firstChild;
cur && cur.nodeName != "BR"; cur = cur.nextSibling)
if (cur.currentText) text.push(cur.currentText);
return {from: node, to: cur, text: cleanText(text.join(""))};
}
// Filter out unchanged lines and nodes that are no longer in the
// document. Build up line objects for remaining nodes.
var lines = [];
if (self.firstTouched) self.touched.push(null);
forEach(self.touched, function(node) {
if (node && node.parentNode != self.container) return;
if (node) node.historyTouched = false;
else self.firstTouched = false;
var line = buildLine(node), shadow = self.after(node);
if (!shadow || shadow.text != line.text || shadow.to != line.to) {
lines.push(line);
setTemp(node, line);
}
});
// Get the BR element after/before the given node.
function nextBR(node, dir) {
var link = dir + "Sibling", search = node[link];
while (search && search.nodeName != "BR")
search = search[link];
return search;
}
// Assemble line objects into chains by scanning the DOM tree
// around them.
var chains = []; self.touched = [];
forEach(lines, function(line) {
// Note that this makes the loop skip line objects that have
// been pulled into chains by lines before them.
if (!temp(line.from)) return;
var chain = [], curNode = line.from, safe = true;
// Put any line objects (referred to by temp info) before this
// one on the front of the array.
while (true) {
var curLine = temp(curNode);
if (!curLine) {
if (safe) break;
else curLine = buildLine(curNode);
}
chain.unshift(curLine);
setTemp(curNode, null);
if (!curNode) break;
safe = self.after(curNode);
curNode = nextBR(curNode, "previous");
}
curNode = line.to; safe = self.before(line.from);
// Add lines after this one at end of array.
while (true) {
if (!curNode) break;
var curLine = temp(curNode);
if (!curLine) {
if (safe) break;
else curLine = buildLine(curNode);
}
chain.push(curLine);
setTemp(curNode, null);
safe = self.before(curNode);
curNode = nextBR(curNode, "next");
}
chains.push(chain);
});
return chains;
},
// Find the 'shadow' of a given chain by following the links in the
// DOM nodes at its start and end.
shadowChain: function(chain) {
var shadows = [], next = this.after(chain[0].from), end = chain[chain.length - 1].to;
while (true) {
shadows.push(next);
var nextNode = next.to;
if (!nextNode || nextNode == end)
break;
else
next = nextNode.historyAfter || this.before(end);
// (The this.before(end) is a hack -- FF sometimes removes
// properties from BR nodes, in which case the best we can hope
// for is to not break.)
}
return shadows;
},
// Update the DOM tree to contain the lines specified in a given
// chain, link this chain into the DOM nodes.
applyChain: function(chain) {
// Some attempt is made to prevent the cursor from jumping
// randomly when an undo or redo happens. It still behaves a bit
// strange sometimes.
var cursor = select.cursorPos(this.container, false), self = this;
// Remove all nodes in the DOM tree between from and to (null for
// start/end of container).
function removeRange(from, to) {
var pos = from ? from.nextSibling : self.container.firstChild;
while (pos != to) {
var temp = pos.nextSibling;
removeElement(pos);
pos = temp;
}
}
var start = chain[0].from, end = chain[chain.length - 1].to;
// Clear the space where this change has to be made.
removeRange(start, end);
// Insert the content specified by the chain into the DOM tree.
for (var i = 0; i < chain.length; i++) {
var line = chain[i];
// The start and end of the space are already correct, but BR
// tags inside it have to be put back.
if (i > 0)
self.container.insertBefore(line.from, end);
// Add the text.
var node = makePartSpan(fixSpaces(line.text), this.container.ownerDocument);
self.container.insertBefore(node, end);
// See if the cursor was on this line. Put it back, adjusting
// for changed line length, if it was.
if (cursor && cursor.node == line.from) {
var cursordiff = 0;
var prev = this.after(line.from);
if (prev && i == chain.length - 1) {
// Only adjust if the cursor is after the unchanged part of
// the line.
for (var match = 0; match < cursor.offset &&
line.text.charAt(match) == prev.text.charAt(match); match++);
if (cursor.offset > match)
cursordiff = line.text.length - prev.text.length;
}
select.setCursorPos(this.container, {node: line.from, offset: Math.max(0, cursor.offset + cursordiff)});
}
// Cursor was in removed line, this is last new line.
else if (cursor && (i == chain.length - 1) && cursor.node && cursor.node.parentNode != this.container) {
select.setCursorPos(this.container, {node: line.from, offset: line.text.length});
}
}
// Anchor the chain in the DOM tree.
this.linkChain(chain);
return start;
}
};

View File

@ -0,0 +1,115 @@
/* A few useful utility functions. */
var internetExplorer = document.selection && window.ActiveXObject && /MSIE/.test(navigator.userAgent);
var webkit = /AppleWebKit/.test(navigator.userAgent);
// Capture a method on an object.
function method(obj, name) {
return function() {obj[name].apply(obj, arguments);};
}
// The value used to signal the end of a sequence in iterators.
var StopIteration = {toString: function() {return "StopIteration"}};
// Apply a function to each element in a sequence.
function forEach(iter, f) {
if (iter.next) {
try {while (true) f(iter.next());}
catch (e) {if (e != StopIteration) throw e;}
}
else {
for (var i = 0; i < iter.length; i++)
f(iter[i]);
}
}
// Map a function over a sequence, producing an array of results.
function map(iter, f) {
var accum = [];
forEach(iter, function(val) {accum.push(f(val));});
return accum;
}
// Create a predicate function that tests a string againsts a given
// regular expression. No longer used but might be used by 3rd party
// parsers.
function matcher(regexp){
return function(value){return regexp.test(value);};
}
// Test whether a DOM node has a certain CSS class. Much faster than
// the MochiKit equivalent, for some reason.
function hasClass(element, className){
var classes = element.className;
return classes && new RegExp("(^| )" + className + "($| )").test(classes);
}
// Insert a DOM node after another node.
function insertAfter(newNode, oldNode) {
var parent = oldNode.parentNode;
parent.insertBefore(newNode, oldNode.nextSibling);
return newNode;
}
function removeElement(node) {
if (node.parentNode)
node.parentNode.removeChild(node);
}
function clearElement(node) {
while (node.firstChild)
node.removeChild(node.firstChild);
}
// Check whether a node is contained in another one.
function isAncestor(node, child) {
while (child = child.parentNode) {
if (node == child)
return true;
}
return false;
}
// The non-breaking space character.
var nbsp = "\u00a0";
var matching = {"{": "}", "[": "]", "(": ")",
"}": "{", "]": "[", ")": "("};
// Standardize a few unportable event properties.
function normalizeEvent(event) {
if (!event.stopPropagation) {
event.stopPropagation = function() {this.cancelBubble = true;};
event.preventDefault = function() {this.returnValue = false;};
}
if (!event.stop) {
event.stop = function() {
this.stopPropagation();
this.preventDefault();
};
}
if (event.type == "keypress") {
event.code = (event.charCode == null) ? event.keyCode : event.charCode;
event.character = String.fromCharCode(event.code);
}
return event;
}
// Portably register event handlers.
function addEventHandler(node, type, handler, removeFunc) {
function wrapHandler(event) {
handler(normalizeEvent(event || window.event));
}
if (typeof node.addEventListener == "function") {
node.addEventListener(type, wrapHandler, false);
if (removeFunc) return function() {node.removeEventListener(type, wrapHandler, false);};
}
else {
node.attachEvent("on" + type, wrapHandler);
if (removeFunc) return function() {node.detachEvent("on" + type, wrapHandler);};
}
}
function nodeText(node) {
return node.innerText || node.textContent || node.nodeValue || "";
}

View File

@ -0,0 +1,56 @@
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<script src="js/codemirror.js" type="text/javascript"></script>
<script src="js/mirrorframe.js" type="text/javascript"></script>
<title>CodeMirror: JavaScript demonstration</title>
<link rel="stylesheet" type="text/css" href="css/docs.css"/>
</head>
<body style="padding: 20px;">
<p>This page demonstrates <a href="index.html">CodeMirror</a>'s
JavaScript parser. Note that the ugly buttons at the top are not are
not part of CodeMirror proper -- they demonstrate the way it can be
embedded in a web-application.</p>
<div class="border">
<textarea id="code" cols="120" rows="30">
// Here you see some JavaScript code. Mess around with it to get
// acquainted with CodeMirror's features.
// Press enter inside the object and your new line will be suitably
// indented.
var keyBindings = {
enter: "newline-and-indent",
tab: "reindent-selection",
ctrl_z: "undo",
ctrl_y: "redo",
ctrl_backspace: "undo-for-safari (which blocks ctrl-z)",
ctrl-bracket: "highlight-brackets",
ctrl-shift-bracket: "jump-to-matching-bracket"
};
// Press tab on the next line and the wrong indentation will be fixed.
var regex = /foo|bar/i;
function example(x) {
// Local variables get a different colour than global ones.
var y = 44.4;
return x + y - z;
}
</textarea>
</div>
<script type="text/javascript">
var textarea = document.getElementById('code');
var editor = new MirrorFrame(CodeMirror.replace(textarea), {
height: "350px",
content: textarea.value,
parserfile: ["tokenizejavascript.js", "parsejavascript.js"],
stylesheet: "css/jscolors.css",
path: "js/",
autoMatchParens: true
});
</script>
</body>
</html>

View File

@ -0,0 +1,52 @@
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<script src="js/codemirror.js" type="text/javascript"></script>
<title>CodeMirror: HTML mixed-mode demonstration</title>
<link rel="stylesheet" type="text/css" href="css/docs.css"/>
</head>
<body style="padding: 20px;">
<p>This is a simple demonstration of the HTML mixed-mode indentation
module for <a href="index.html">CodeMirror</a>. Script tags use the JS
parser, style tags use the CSS parser.</p>
<div style="border: 1px solid black; padding: 3px;">
<textarea id="code" cols="120" rows="30">
&lt;html>
&lt;head>
&lt;title>HTML Example&lt;/title>
&lt;script type="text/javascript">
function foo(bar, baz) {
alert("quux");
return bar + baz + 1;
}
&lt;/script>
&lt;style type="text/css">
div.border {
border: 1px solid black;
padding: 3px;
}
#foo code {
font-family: courier, monospace;
font-size: 80%;
color: #448888;
}
&lt;/style>
&lt;/head>
&lt;body>
&lt;p>Duh&lt;/p>
&lt;/body>
&lt;/html>
</textarea>
</div>
<script type="text/javascript">
var editor = CodeMirror.fromTextArea('code', {
height: "350px",
parserfile: ["parsexml.js", "parsecss.js", "tokenizejavascript.js", "parsejavascript.js", "parsehtmlmixed.js"],
stylesheet: ["css/xmlcolors.css", "css/jscolors.css", "css/csscolors.css"],
path: "js/"
});
</script>
</body>
</html>

View File

@ -0,0 +1,41 @@
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<script src="js/codemirror.js" type="text/javascript"></script>
<title>CodeMirror: Sparql demonstration</title>
<link rel="stylesheet" type="text/css" href="css/docs.css"/>
</head>
<body style="padding: 20px;">
<p>Demonstration of <a href="index.html">CodeMirror</a>'s Sparql
highlighter.</p>
<div class="border">
<textarea id="code" cols="120" rows="30">
PREFIX a: &lt;http://www.w3.org/2000/10/annotation-ns#>
PREFIX dc: &lt;http://purl.org/dc/elements/1.1/>
PREFIX foaf: &lt;http://xmlns.com/foaf/0.1/>
# Comment!
SELECT ?given ?family
WHERE {
?annot a:annotates &lt;http://www.w3.org/TR/rdf-sparql-query/> .
?annot dc:creator ?c .
OPTIONAL {?c foaf:given ?given ;
foaf:family ?family } .
FILTER isBlank(?c)
}
</textarea>
</div>
<script type="text/javascript">
var editor = CodeMirror.fromTextArea('code', {
height: "250px",
parserfile: "parsesparql.js",
stylesheet: "css/sparqlcolors.css",
path: "js/"
});
</script>
</body>
</html>

View File

@ -0,0 +1,652 @@
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<title>Implementing a syntax-higlighting JavaScript editor in JavaScript</title>
<style type="text/css">
body {
padding: 3em 6em;
max-width: 50em;
}
h1 {
text-align: center;
margin: 0;
}
h2 {
font-size: 130%;
}
code {
font-family: courier, monospace;
font-size: 80%;
color: #144;
}
p {
margin: 1em 0;
}
pre.code {
margin: 1.1em 12px;
border: 1px solid #CCCCCC;
padding: .4em;
font-family: courier, monospace;
}
</style>
<link rel="stylesheet" type="text/css" href="css/jscolors.css"/>
</head>
<body>
<h1 style="font-size: 180%;">Implementing a syntax-higlighting JavaScript editor in JavaScript</h1>
<h1 style="font-size: 110%;">or</h1>
<h1 style="font-size: 130%; margin-bottom: 3em;">A brutal odyssey to the dark side of the DOM tree</h1>
<p style="font-size: 80%">
<b>Topic</b>: JavaScript, advanced browser weirdness, cool programming techniques<br/>
<b>Audience</b>: Programmers, especially JavaScript programmers<br/>
<b>Author</b>: Marijn Haverbeke<br/>
<b>Date</b>: May 24th 2007
</p>
<p style="color: #811; font-size: 90%; font-style: italic">Note: some of the details given here no
longer apply to the current <a
href="http://marijn.haverbeke.nl/codemirror">CodeMirror</a>
codebase, which has evolved quite a bit in the meantime.</p>
<p>In one of his (very informative) <a
href="http://www.learnwebdesignonline.com/videos/programming/javascript/yahoo-douglas-crockford">video
lectures</a>, Douglas Crockford remarks that writing JavaScript
for the web is 'programming in a hostile environment'. I had done
my fair share of weird workarounds, and even occasonally gave up
an on idea entirely because browsers just wouldn't support it, but
before this project I never really realized just how powerless a
programmer can be in the face of buggy, incompatible, and poorly
designed platforms.</p>
<p>The plan was not ridiculously ambitious. I wanted to 'enhance' a
textarea to the point where writing code in it is pleasant. This meant
automatic indentation and, if possible at all, syntax highlighting.</p>
<p>In this document I describe the story of implementing this, for your
education and amusement. A demonstration of the resulting program,
along with the source code, can be found at <a
href="http://marijn.haverbeke.nl/codemirror">my website</a>.</p>
<h2>Take one: Only indentation</h2>
<p>The very first attempt merely added auto-indentation to a textarea
element. It would scan backwards through the content of the area,
starting from the cursor, until it had enough information to decide
how to indent the current line. It took me a while to figure out a
decent model for indenting JavaScript code, but in the end this seems
to work:</p>
<ul>
<li>Code that sits inside a block is indented one unit (generally two
spaces) more than the statement or brace that opened the block.</li>
<li>A statement that is continued to the next line is indented one unit
more than the line that starts the statement.</li>
<li>When dealing with lists of arguments or the content of array and
object literals there are two possible models. If there is any text
directly after the opening brace, bracket, or parenthesis,
subsequent lines are aligned with this opening character. If the
opening character is followed by a newline (optionally with whitespace
or comments before it), the next line is indented one unit further
than the line that started the list.</li>
<li>And, obviously, if a statement follows another statement it is
indented the same amount as the one before it.</li>
</ul>
<p>When scanning backwards through code one has to take string values,
comments, and regular expressions (which are delimited by slashes)
into account, because braces and semicolons and such are not
significant when they appear inside them. Single-line ('//') comments
turned out to be rather inefficient to check for when doing a
backwards scan, since every time you encounter a newline you have to
go on to the next newline to determine whether this line ends in a
comment or not. Regular expressions are even worse &#x2015; without
contextual information they are impossible to distinguish from the
division operator, and I didn't get them working in this first
version.</p>
<p>To find out which line to indent, and to make sure that adding or
removing whitespace doesn't cause the cursor to jump in strange ways,
it is necessary to determine which text the user has selected. Even
though I was working with just a simple textarea at this point, this
was already a bit of a headache.</p>
<p>On W3C-standards-respecting browsers, textarea nodes have
<code>selectionStart</code> and <code>selectionEnd</code>
properties which nicely give you the amount of characters before
the start and end of the selection. Great!</p>
<p>Then, there is Internet Explorer. Internet Explorer also has an API
for looking at and manipulating selections. It gives you information
such as a detailed map of the space the selected lines take up on the
screen, in pixels, and of course the text inside the selection. It
does, however, not give you much of a clue on where the selection is
located in the document.</p>
<p>After some experimentation I managed to work out an elaborate
method for getting something similar to the
<code>selectionStart</code> and <code>selectionEnd</code> values
in other browsers. It worked like this:</p>
<ul>
<li>Get the <code>TextRange</code> object corresponding to the selection.</li>
<li>Record the length of the text inside it.</li>
<li>Make another <code>TextRange</code> that covers the whole textarea element.</li>
<li>Set the start of the first <code>TextRange</code> to the start of the second one.</li>
<li>Again get the length of the text in the first object.</li>
<li>Now <code>selectionEnd</code> is the second length, and <code>selectionStart</code> is
the second minus the first one.</li>
</ul>
<p>That seemed to work, but when resetting the selection after modifying
the content of the textarea I ran into another interesting feature of
these <code>TextRange</code>s: You can move their endpoints by a given number of
characters, which is useful when trying to set a cursor at the Nth
character of a textarea, but in this context, newlines are <em>not</em>
considered to be characters, so you'll always end up one character too
far for every newline you passed. Of course, you can count newlines
and compensate for this (though it is still not possible to position
the cursor right in front of a newline). Sheesh.</p>
<p>After ragging on Internet Explorer for a while, let us move on and rag
on Firefox a bit. It turns out that, in Firefox, getting and setting
the text content of a DOM element is unexplainably expensive,
especially when there is a lot of text involved. As soon as I tried to
use my indentation code to indent itself (some 400 lines), I found
myself waiting for over four seconds every time I pressed enter. That
seemed a little slow.</p>
<h2>designMode it is</h2>
<p>The solution was obvious: Since the text inside a textarea can only be
manipulated as one single big string, I had to spread it out over
multiple nodes. How do you spread editable content over multiple
nodes? Right! <code>designMode</code> or <code>contentEditable</code>.</p>
<p>Now I wasn't entirely naive about <code>designMode</code>, I had been looking
into writing a non-messy WYSIWYG editor before, and at that time I had
concluded two things:</p>
<ul>
<li>It is impossible to prevent the user from inserting whichever HTML
junk he wants into the document.</li>
<li>In Internet Explorer, it is extemely hard to get a good view
on what nodes the user has selected.</li>
</ul>
<p>Basically, the good folks at Microsoft designed a really bad interface
for putting editable documents in pages, and the other browsers, not
wanting to be left behind, more or less copied that. And there isn't
much hope for a better way to do this appearing anytime soon. Wise
people probably use a Flash movie or (God forbid) a Java applet for
these kind of things, though those are not without drawbacks either.</p>
<p>Anyway, seeing how using an editable document would also make syntax
highlighting possible, I foolishly went ahead. There is something
perversely fascinating about trying to build a complicated system on a
lousy, unsuitable platform.</p>
<h2>A parser</h2>
<p>How does one do decent syntax highlighting? A very simple scanning can
tell the difference between strings, comments, keywords, and other
code. But this time I wanted to actually be able to recognize regular
expressions, so that I didn't have any blatant incorrect behaviour
anymore.</p>
<p>That brought me to the idea of doing a serious parse on the code. This
would not only make detecting regular expressions much easier, it
would also give me detailed information about the code, which can be
used to determine proper indentation levels, and to make subtle
distinctions in colouring, for example the difference between variable
names and property names.</p>
<p>And hey, when we're parsing the whole thing, it would even be possible
to make a distinction between local and global variables, and colour
them differently. If you've ever programmed JavaScript you can
probably imagine how useful this would be &#x2015; it is ridiculously easy
to accidentally create global instead of local variables. I don't
consider myself a JavaScript rookie anymore, but it was (embarrasingly
enough) only this week that I realized that my habit of typing <code>for
(name in object) ...</code> was creating a global variable <code>name</code>, and that
I should be typing <code>for (var name in object) ...</code> instead.</p>
<p>Re-parsing all the code the user has typed in every time he hits a key
is obviously not feasible. So how does one combine on-the-fly
highlighting with a serious parser? One option would be to split the
code into top-level statements (functions, variable definitions, etc.)
and parse these separately. This is horribly clunky though, especially
considering the fact that modern JavaScripters often put all the code
in a file in a single big object or function to prevent namespace
pollution.</p>
<p>I have always liked continuation-passing style and generators. So the
idea I came up with is this: An interruptable, resumable parser. This
is a parser that does not run through a whole document at once, but
parses on-demand, a little bit at a time. At any moment you can create
a copy of its current state, which can be resumed later. You start
parsing at the top of the code, and keep going as long as you like,
but throughout the document, for example at every end of line, you
store a copy of the current parser state. Later on, when line 106
changes, you grab the interrupted parser that was stored at the end of
line 105, and use it to re-parse line 106. It still knows exactly what
the context was at that point, which local variables were defined,
which unfinished statements were encountered, and so on.</p>
<p>But that, unfortunately, turned out to be not quite as easy as it
sounds.</p>
<h2>The DOM nodes underfoot</h2>
<p>Of course, when working inside an editable frame we don't just
have to deal with text. The code will be represented by some kind
of DOM tree. My first idea was to set the <code>white-space:
pre</code> style for the frame and try to work with mostly text,
with the occasional coloured <code>span</code> element. It turned
out that support for <code>white-space: pre</code> in browsers,
especially in editable frames, is so hopelessly glitchy that this
was unworkable.</p>
<p>Next I tried a series of <code>div</code> elements, one per
line, with <code>span</code> elements inside them. This seemed to
nicely reflect the structure of the code in a shallowly
hierarchical way. I soon realized, however, that my code would be
much more straightfoward when using no hierarchy whatsoever
&#x2015; a series of <code>span</code>s, with <code>br</code> tags
at the end of every line. This way, the DOM nodes form a flat
sequence that corresponds to the sequence of the text &#x2015;
just extract text from <code>span</code> nodes and substitute
newlines for <code>br</code> nodes.</p>
<p>It would be a shame if the editor would fall apart as soon as
someone pastes some complicated HTML into it. I wanted it to be
able to deal with whatever mess it finds. This means using some
kind of HTML-normalizer that takes arbitrary HTML and flattens it
into a series of <code>br</code>s and <code>span</code> elements
that contain a single text node. Just like the parsing process, it
would be best if this did not have to done to the entire buffer
every time something changes.</p>
<p>It took some banging my head against my keyboard, but I found a very
nice way to model this. It makes heavy use of generators, for which I
used <a href="http://www.mochikit.com">MochiKit</a>'s iterator
framework. Bob Ippolito explains the concepts in this library very
well in his <a
href="http://bob.pythonmac.org/archives/2005/07/06/iteration-in-javascript/">blog
post</a> about it. (Also notice some of the dismissive comments at the
bottom of that post. They say "I don't think I really want to learn
this, so I'll make up some silly reason to condemn it.")</p>
<p>The highlighting process consists of the following elements:
normalizing the DOM tree, extracting the text from the DOM tree,
tokenizing this text, parsing the tokens, and finally adjusting the
DOM nodes to reflect the structure of the code.</p>
<p>The first two, I put into a single generator. It scans the DOM
tree, fixing anything that is not a simple top-level
<code>span</code> or <code>br</code>, and it produces the text
content of the nodes (or a newline in case of a <code>br</code>)
as its output &#x2015; each time it is called, it yields a string.
Continuation passing style was a good way to model this process in
an iterator, which has to be processed one step at a time. Look at
this simplified version:</p>
<pre class="code"><span class="js-keyword">function</span> <span class="js-variable">traverseDOM</span>(<span class="js-variabledef">start</span>){
<span class="js-keyword">var</span> <span class="js-variabledef">cc</span> = <span class="js-keyword">function</span>(){<span class="js-variable">scanNode</span>(<span class="js-localvariable">start</span>, <span class="js-variable">stop</span>);};
<span class="js-keyword">function</span> <span class="js-variabledef">stop</span>(){
<span class="js-localvariable">cc</span> = <span class="js-localvariable">stop</span>;
<span class="js-keyword">throw</span> <span class="js-variable">StopIteration</span>;
}
<span class="js-keyword">function</span> <span class="js-variabledef">yield</span>(<span class="js-variabledef">value</span>, <span class="js-variabledef">c</span>){
<span class="js-localvariable">cc</span> = <span class="js-localvariable">c</span>;
<span class="js-keyword">return</span> <span class="js-localvariable">value</span>;
}
<span class="js-keyword">function</span> <span class="js-variabledef">scanNode</span>(<span class="js-variabledef">node</span>, <span class="js-variabledef">c</span>){
<span class="js-keyword">if</span> (<span class="js-localvariable">node</span>.<span class="js-property">nextSibling</span>)
<span class="js-keyword">var</span> <span class="js-variabledef">nextc</span> = <span class="js-keyword">function</span>(){<span class="js-localvariable">scanNode</span>(<span class="js-localvariable">node</span>.<span class="js-property">nextSibling</span>, <span class="js-localvariable">c</span>);};
<span class="js-keyword">else</span>
<span class="js-keyword">var</span> <span class="js-variabledef">nextc</span> = <span class="js-localvariable">c</span>;
<span class="js-keyword">if</span> (<span class="js-comment">/* node is proper span element */</span>)
<span class="js-keyword">return</span> <span class="js-localvariable">yield</span>(<span class="js-localvariable">node</span>.<span class="js-property">firstChild</span>.<span class="js-property">nodeValue</span>, <span class="js-localvariable">nextc</span>);
<span class="js-keyword">else</span> <span class="js-keyword">if</span> (<span class="js-comment">/* node is proper br element */</span>)
<span class="js-keyword">return</span> <span class="js-localvariable">yield</span>(<span class="js-string">&quot;\n&quot;</span>, <span class="js-localvariable">nextc</span>);
<span class="js-keyword">else</span>
<span class="js-comment">/* flatten node, yield its textual content */</span>;
}
<span class="js-keyword">return</span> {<span class="js-property">next</span>: <span class="js-keyword">function</span>(){<span class="js-keyword">return</span> <span class="js-localvariable">cc</span>();}};
}</pre>
<p>The variable <code>c</code> stands for 'continuation', and <code>cc</code> for 'current
continuation' &#x2015; that last variable is used to store the function to
continue with, when yielding a value to the outside world. Every time
control leaves this function, it has to make sure that <code>cc</code> is set to
a suitable value, which is what <code>yield</code> and <code>stop</code> take care of.</p>
<p>The object that is returned contains a <code>next</code> method, which is
MochiKit's idea of an iterator, and the initial continuation just
throws a <code>StopIteration</code>, which is how MochiKit signals that an
iterator has reached its end.</p>
<p>The first lines of <code>scanNode</code> extend the continuation with the task of
scanning the next node, if there is a next node. The rest of the
function decides what kind of value to <code>yield</code>. Note that this is a
rather trivial example of this technique, since the process of going
through these nodes is basically linear (it was much, much more
complex in earlier versions), but still the trick with the
continuations makes the code shorter and, for those in the know,
clearer than the equivalent 'storing the iterator state in variables'
approach.</p>
<p>The next iterator that the input passes through is the
tokenizer. Well, actually, there is another iterator in between
that isolates the tokenizer from the fact that the DOM traversal
yields a bunch of separate strings, and presents them as a single
character stream (with a convenient <code>peek</code> operation),
but this is not a very interesting one. What the tokenizer returns
is a stream of token objects, each of which has a
<code>value</code>, its textual content, a <code>type</code>, like
<code>"variable"</code>, <code>"operator"</code>, or just itself,
<code>"{"</code> for example, in the case of significant
punctuation or special keywords. They also have a
<code>style</code>, which is used later by the highlighter to give
their <code>span</code> elements a class name (the parser will
still adjust this in some cases).</p>
<p>At first I assumed the parser would have to talk back to the
tokenizer about the current context, in order to be able to
distinguish those accursed regular expressions from divisions, but
it seems that regular expressions are only allowed if the previous
(non-whitespace, non-comment) token was either an operator, a
keyword like <code>new</code> or <code>throw</code>, or a specific
kind of punctuation (<code>"[{}(,;:"</code>) that indicates a new
expression can be started here. This made things considerably
easier, since the 'regexp or no regexp' question could stay
entirely within the tokenizer.</p>
<p>The next step, then, is the parser. It does not do a very
thorough job because, firstly, it has to be fast, and secondly, it
should not go to pieces when fed an incorrect program. So only
superficial constructs are recognized, keywords that resemble each
other in syntax, such as <code>while</code> and <code>if</code>,
are treated in precisely the same way, as are <code>try</code> and
<code>else</code> &#x2015; the parser doesn't mind if an
<code>else</code> appears without an <code>if</code>. Stuff that
binds variables, <code>var</code>, <code>function</code>, and
<code>catch</code> to be precise, is treated with more care,
because the parser wants to know about local variables.</p>
<p>Inside the parser, three kinds of context are stored. Firstly, a set
of known local variables, which is used to adjust the style of
variable tokens. Every time the parser enters a function, a new set of
variables is created. If there was already such a set (entering an
inner function), a pointer to the old one is stored in the new one. At
the end of the function, the current variable set is 'popped' off and
the previous one is restored.</p>
<p>The second kind of context is the lexical context, this keeps track of
whether we are inside a statement, block, or list. Like the variable
context, it also forms a stack of contexts, with each one containing a
pointer to the previous ones so that they can be popped off again when
they are finished. This information is used for indentation. Every
time the parser encounters a newline token, it attaches the current
lexical context and a 'copy' of itself (more about that later) to this
token.</p>
<p>The third context is a continuation context. This parser does not use
straight continuation style, instead it uses a stack of actions that
have to be performed. These actions are simple functions, a kind of
minilanguage, they act on tokens, and decide what kind of new actions
should be pushed onto the stack. Here are some examples:</p>
<pre class="code"><span class="js-keyword">function</span> <span class="js-variable">expression</span>(<span class="js-variabledef">type</span>){
<span class="js-keyword">if</span> (<span class="js-localvariable">type</span> in <span class="js-variable">atomicTypes</span>) <span class="js-variable">cont</span>(<span class="js-variable">maybeoperator</span>);
<span class="js-keyword">else</span> <span class="js-keyword">if</span> (<span class="js-localvariable">type</span> == <span class="js-string">&quot;function&quot;</span>) <span class="js-variable">cont</span>(<span class="js-variable">functiondef</span>);
<span class="js-keyword">else</span> <span class="js-keyword">if</span> (<span class="js-localvariable">type</span> == <span class="js-string">&quot;(&quot;</span>) <span class="js-variable">cont</span>(<span class="js-variable">pushlex</span>(<span class="js-string">&quot;list&quot;</span>), <span class="js-variable">expression</span>, <span class="js-variable">expect</span>(<span class="js-string">&quot;)&quot;</span>), <span class="js-variable">poplex</span>);
<span class="js-keyword">else</span> <span class="js-keyword">if</span> (<span class="js-localvariable">type</span> == <span class="js-string">&quot;operator&quot;</span>) <span class="js-variable">cont</span>(<span class="js-variable">expression</span>);
<span class="js-keyword">else</span> <span class="js-keyword">if</span> (<span class="js-localvariable">type</span> == <span class="js-string">&quot;[&quot;</span>) <span class="js-variable">cont</span>(<span class="js-variable">pushlex</span>(<span class="js-string">&quot;list&quot;</span>), <span class="js-variable">commasep</span>(<span class="js-variable">expression</span>), <span class="js-variable">expect</span>(<span class="js-string">&quot;]&quot;</span>), <span class="js-variable">poplex</span>);
<span class="js-keyword">else</span> <span class="js-keyword">if</span> (<span class="js-localvariable">type</span> == <span class="js-string">&quot;{&quot;</span>) <span class="js-variable">cont</span>(<span class="js-variable">pushlex</span>(<span class="js-string">&quot;list&quot;</span>), <span class="js-variable">commasep</span>(<span class="js-variable">objprop</span>), <span class="js-variable">expect</span>(<span class="js-string">&quot;}&quot;</span>), <span class="js-variable">poplex</span>);
<span class="js-keyword">else</span> <span class="js-keyword">if</span> (<span class="js-localvariable">type</span> == <span class="js-string">&quot;keyword c&quot;</span>) <span class="js-variable">cont</span>(<span class="js-variable">expression</span>);
}
<span class="js-keyword">function</span> <span class="js-variable">block</span>(<span class="js-variabledef">type</span>){
<span class="js-keyword">if</span> (<span class="js-localvariable">type</span> == <span class="js-string">&quot;}&quot;</span>) <span class="js-variable">cont</span>();
<span class="js-keyword">else</span> <span class="js-variable">pass</span>(<span class="js-variable">statement</span>, <span class="js-variable">block</span>);
}</pre>
<p>The function <code>cont</code> (for continue), will push the actions it is given
onto the stack (in reverse order, so that the first one will be popped
first). Actions such as <code>pushlex</code> and <code>poplex</code> merely adjust the
lexical environment, while others, such as <code>expression</code> itself, do
actual parsing. <code>pass</code>, as seen in <code>block</code>, is similar to <code>cont</code>, but
it does not 'consume' the current token, so the next action will again
see this same token. In <code>block</code>, this happens when the function
determines that we are not at the end of the block yet, so it pushes
the <code>statement</code> function which will interpret the current token as the
start of a statement.</p>
<p>These actions are called by a 'driver' function, which filters out the
whitespace and comments, so that the parser actions do not have to
think about those, and keeps track of some things like the indentation
of the current line and the column at which the current token ends,
which are stored in the lexical context and used for indentation.
After calling an action, if the action called <code>cont</code>, this driver
function will return the current token, if <code>pass</code> (or nothing) was
called, it will immediately continue with the next action.</p>
<p>This goes to show that it is viable to write a quite elaborate
minilanguage in a macro-less language like JavaScript. I don't think
it would be possible to do something like this without closures (or
similarly powerful abstraction) though, I've certainly never seen
anything like it in Java code.</p>
<p>The way a 'copy' of the parser was produced shows a nice usage
of closures. Like with the DOM transformer shown above, most of
the local state of the parser is held in a closure produced by
calling <code>parse(stream)</code>. The function
<code>copy</code>, which is local to the parser function, produces
a new closure, with copies of all the relevant variables:</p>
<pre class="code"><span class="js-keyword">function</span> <span class="js-variable">copy</span>(){
<span class="js-keyword">var</span> <span class="js-variabledef">_context</span> = <span class="js-variable">context</span>, <span class="js-variabledef">_lexical</span> = <span class="js-variable">lexical</span>, <span class="js-variabledef">_actions</span> = <span class="js-variable">copyArray</span>(<span class="js-variable">actions</span>);
<span class="js-keyword">return</span> <span class="js-keyword">function</span>(<span class="js-variabledef">_tokens</span>){
<span class="js-variable">context</span> = <span class="js-localvariable">_context</span>;
<span class="js-variable">lexical</span> = <span class="js-localvariable">_lexical</span>;
<span class="js-variable">actions</span> = <span class="js-variable">copyArray</span>(<span class="js-localvariable">_actions</span>);
<span class="js-variable">tokens</span> = <span class="js-localvariable">_tokens</span>;
<span class="js-keyword">return</span> <span class="js-variable">parser</span>;
};
}</pre>
<p>Where <code>parser</code> is the object that contains the <code>next</code> (driver)
function, and a reference to this <code>copy</code> function. When the function
that <code>copy</code> produces is called with a token stream as argument, it
updates the local variables in the parser closure, and returns the
corresponding iterator object.</p>
<p>Moving on, we get to the last stop in this chain of generators, the
actual highlighter. You can view this one as taking two streams as
input, on the one hand there is the stream of tokens from the parser,
and on the other hand there is the DOM tree as left by the DOM
transformer. If everything went correctly, these two should be
synchronized. The highlighter can look at the current token, see if
the <code>span</code> in the DOM tree corresponds to it (has the same text
content, and the correct class), and if not it can chop up the DOM
nodes to conform to the tokens.</p>
<p>Every time the parser yields a newline token, the highligher
encounters a <code>br</code> element in the DOM stream. It takes the copy of the
parser and the lexical context from this token and attaches them to
the DOM node. This way, a new highlighting process can be started from
that node by re-starting the copy of the parser with a new token
stream, which reads tokens from the DOM nodes starting at that <code>br</code>
element, and the indentation code can use the lexical context
information to determine the correct indentation at that point.</p>
<h2>Selection woes</h2>
<p>All the above can be done using the DOM interface that all major
browsers have in common, and which is relatively free of weird bugs
and abberrations. However, when the user is typing in new code, this
must also be highlighted. For this to happen, the program must know
where the cursor currently is, and because it mucks up the DOM tree,
it has to restore this cursor position after doing the highlighting.</p>
<p>Re-highlighting always happens per line, because the copy of the
parser is stored only at the end of lines. Doing this every time the
user presses a key is terribly slow and obnoxious, so what I did was
keep a list of 'dirty' nodes, and as soon as the user didn't type
anyting for 300 milliseconds the program starts re-highlighting these
nodes. If it finds more than ten lines must be re-parsed, it does only
ten and waits another 300 milliseconds before it continues, this way
the browser never freezes up entirely.</p>
<p>As mentioned earlier, Internet Explorer's selection model is not the
most practical one. My attempts to build a wrapper that makes it look
like the W3C model all stranded. In the end I came to the conclusion
that I only needed two operations:</p>
<ul>
<li>Creating a selection 'snapshot' that can be restored after
highlighting, in such a way that it still works if some of the nodes
that were selected are replaced by other nodes with the same
size but a different structure.</li>
<li>Finding the top-level node around or before the cursor, to mark it
dirty or to insert indentation whitespace at the start of that line.</li>
</ul>
<p>It turns out that the pixel-based selection model that Internet
Explorer uses, which always seemed completely ludricrous to me, is
perfect for the first case. Since the DOM transformation (generally)
does not change the position of things, storing the pixel offsets of
the selection makes it possible to restore that same selection, never
mind what happened to the underlying DOM structure.</p>
<p>[Later addition: Note that this, due to the very random design
of the <a
href="http://msdn2.microsoft.com/en-us/library/ms535872(VS.85).aspx#">TextRange
interface</a>, only really works when the whole selection falls
within the visible part of the document.]</p>
<p>Doing the same with the W3C selection model is a lot harder. What I
ended up with was this:</p>
<ul>
<li>Create an object pointing to the nodes at the start and end of the
selection, and the offset within those nodes. This is basically the
information that the <code>Range</code> object gives you.</li>
<li>Make references from these nodes back to that object.</li>
<li>When replacing (part of) a node with another one, check for such a
reference, and when it is present, check whether this new node will
get the selection. If it does, move the reference from the old to the
new node, if it does not, adjust the offset in the selection object to
reflect the fact that part of the old node has been replaced.</li>
</ul>
<p>Now in the second case (getting the top-level node at the
cursor) the Internet Explorer cheat does not work. In the W3C
model this is rather easy, you have to do some creative parent-
and sibling-pointer following to arrive at the correct top-level
node, but nothing weird. In Internet Explorer, all we have to go
on is the <code>parentElement</code> method on a
<code>TextRange</code>, which gives the first element that
completely envelops the selection. If the cursor is inside a text
node, this is good, that text node tells us where we are. If the
cursor is between nodes, for example between two <code>br</code>
nodes, you get to top-level node itself back, which is remarkably
useless. In cases like this I stoop to a rather ugly hack (which
fortunately turned out to be acceptably fast) &#x2015; I create a
temporary empty <code>span</code> with an ID inside the selection,
get a reference to this <code>span</code> by ID, take its
<code>previousSibling</code>, and remove it again.</p>
<p>Unfortunately, Opera's selection implementation is buggy, and it
will give wildly incorrect <code>Range</code> objects when the cursor
is between two nodes. This is a bit of a showstopper, and until I find
a workaround for that or it gets fixed, the highlighter doesn't work
properly in Opera.</p>
<p>Also, when one presses enter in a <code>designMode</code>
document in Firefox or Opera, a <code>br</code> tag is inserted.
In Internet Explorer, pressing enter causes some maniacal gnome to
come out and start wrapping all the content before and after the
cursor in <code>p</code> tags. I suppose there is something to be
said for that, in principle, though if you saw the tag soup of
<code>font</code>s and nested paragraphs Internet Explorer
generates you would soon enough forget all about principle.
Anyway, getting unwanted <code>p</code> tags slowed the
highlighter down terribly &#x2015; it had to overhaul the whole
DOM tree to remove them again, every time the user pressed enter.
Fortunately I could fix this by capturing the enter presses and
manually inserting a <code>br</code> tag at the cursor.</p>
<p>On the subject of Internet Explorer's tag soup, here is an interesting
anecdote: One time, when testing the effect that modifying the content
of a selection had, I inspected the DOM tree and found a <code>"/B"</code>
element. This was not a closing tag, there are no closing tags in the
DOM tree, just elements. The <code>nodeName</code> of this element was actually
<code>"/B"</code>. That was when I gave up any notions of ever understanding the
profound mystery that is Internet Explorer.</p>
<h2>Closing thoughts</h2>
<p>Well, I despaired at times, but I did end up with a working JavaScript
editor. I did not keep track of the amount of time I wasted on this,
but I would estimate it to be around fifty hours. Finding workarounds
for browser bugs can be a terribly nonlinear process. I just spent
half a day working on a weird glitch in Firefox that caused the cursor
in the editable frame to be displayed 3/4 line too high when it was at
the very end of the document. Then I found out that setting the
style.display of the iframe to "block" fixed this (why not?). I'm
amazed how often issues that seem hopeless do turn out to be
avoidable, even if it takes hours of screwing around and some truly
non-obvious ideas.</p>
<p>For a lot of things, JavaScript + DOM elements are a surprisingly
powerful platform. Simple interactive documents and forms can be
written in browsers with very little effort, generally less than with
most 'traditional' platforms (Java, Win32, things like WxWidgets).
Libraries like Dojo (and a similar monster I once wrote myself) even
make complex, composite widgets workable. However, when applications
go sufficiently beyond the things that browsers were designed for, the
available APIs do not give enough control, are nonstandard and buggy,
and are often poorly designed. Because of this, writing such
applications, when it is even possible, is <em>painful</em> process.</p>
<p>And who likes pain? Sure, when finding that crazy workaround,
subdueing the damn browser, and getting everything to work, there
is a certain macho thrill. But one can't help wondering how much
easier things like preventing the user from pasting pictures in
his source code would be on another platform. Maybe something like
Silverlight or whatever other new browser plugin gizmos people are
pushing these days will become the way to solve things like this
in the future. But, personally, I would prefer for those browser
companies to put some real effort into things like cleaning up and
standardising shady things like <code>designMode</code>, fixing
their bugs, and getting serious about ECMAScript 4.</p>
<p>Which is probably not realistically going to happen anytime soon.</p>
<hr/>
<p>Some interesting projects similar to this:</p>
<ul>
<li><a href="http://gpl.internetconnection.net/vi/">vi clone</a></li>
<li><a href="http://robrohan.com/projects/9ne/">Emacs clone</a></li>
<li><a href="http://codepress.sourceforge.net/">CodePress</a></li>
<li><a href="http://www.codeide.com">CodeIDE</a></li>
<li><a href="http://www.cdolivet.net/editarea">EditArea</a></li>
</ul>
<hr/>
<p>If you have any remarks, criticism, or hints related to the
above, drop me an e-mail at <a
href="mailto:marijnh@gmail.com">marijnh@gmail.com</a>. If you say
something generally interesting, I'll include your reaction here
at the bottom of this page.</p>
</body>
</html>