dos batch copy directory contents with wild card

28 September 2014

If you want to use xcopy in a batchfile to mirror the contents of some other path with a wildcard expression, instead of dumping all the files into your current working directory, this may be what you need:

for /D %%I in (%1) do xcopy “%%I” “%%~nI” /S /I

If you think of the fully qualified pathname as a slash-delimited stack, the ~n operator pops the last element from it and uses it as the target parameter for xcopy. If you put this in a batchfile called, say, copys, you can use like so:

copys d:\some\network\path\backup*

and your current working directory will get the backup* directories and their contents.

word frequency count

25 December 2011

cat {file} |tr [:upper:] [:lower:]\
| sed ’s/[^[:alpha:]][^[:alpha:]]*/\
/g’ | sort | uniq -c| sort -r |more

note that a new line is explicitly included in the sed via the line break. . .

thanks to Daniel Fackrell for the inspiration

javascript open tab browser link

10 July 2008

Someone just asked me how to open a link in a new tab with JavaScript (in Firefox). I thought it couldn’t be done. Went looking for confirmation, and discovered that it can be done, sort of. This post by Larry Williams from March 2005 explains how to open a new tab with JavaScript (from an extension). However the current top-ranked link to this post on Google inexplicably points to a non-existent entry from January 2005 instead. I’m posting this spider-friendly corrected pointer as a public service announcement. You’re welcome.

pluginreg.dat + flash + DO NOT WANT

17 June 2008

I don’t think I’m being all that unreasonable. Here’s what I want:

  • Opera has superior tools for maintaining privacy. I use Opera for all the things people send me links to. I don’t mind installing plugins, within reason, including Flash, since I can easily clean up after each browsing session.
  • Firefox is the browser that I use for anything transactional. I allow persistent cookies from trusted sites. I don’t want plugins installed. (Mozilla itself says “Plug-ins can cause crashes and hangs and may contain security vulnerabilities”).

Last time I updated the Flash player in Opera, it installed itself into Firefox’s plugin directory as well. I was not pleased, but the fix was simple: in . . ./Firefox/plugins I renamed NPSFW32.dll to NPSFW32.DONOTWANT.DLL and all was well.

With the latest update, however, the Flash installer doesn’t copy the DLLs to Firefox’s plugin directory — it puts a pointer to the version in Opera in Firefox’s pluginreg.dat:

...\Opera\program\plugins\NPSWF32.dll|$

So I change it to

...\Opera\program\DONOTWANT\NPSWF32.dll|$

. . .and the next time I restart Firefox, pluginreg.dat is re-written from scratch, and Flash is back.

Next thing I try is to see where the data that generates pluginreg.dat is. This is not straightforward. I can’t find NPSFW32.dll as a plaintext string anywhere obvious.

Next attempt seemed to work at first but didn’t. I made pluginreg.dat a read-only file. Firefox currently respects its read-only status and leaves my DONOTWANT edit intact.

Update: turns out pluginreg.dat sounds promising but is just a red herring. (Adobe has a Flash version test page so it’s easy to tell.) Even with a DONOTWANT string in pluginreg.dat, Firefox still finds the plugin somehow. Argh!

Next I notice that opera’s ini file specifies a plugin directory instead of specific plugins.

Aha! I can rename the flash DLL to something like _NPSFW32.dll. Opera loads it fine. Firefox can’t find it, and removes the entry from pluginreg.dat completely. Success, for real . . . for now.

renner - hacky little file renaming utility

12 November 2007

Even in these days of graphical development environments, I often find myself needing to change the extensions of hundreds of files, or remove “_converted” from the end of a file name, or zero-pad the beginning of a file name so “001″ “2″ and “03″ will sort correctly.

Renner is a little perl script I hacked together to do all those tasks. It’s not pretty, but it’s gotten the job done for me for years. Mostly I’m putting it on the Internets so it’s easy for me to find it when I need it.

How to use it:

Summary Help

perl renner.pl location [matchcriterion] action [utility]
S = a string
n = a number (of characters)
match criteria:
-mask S|-unmask S
-dir
actions:prepend/append/unprepend/unappend(truncate):
-app S| -pre S| -unapp n| -unpre n
actions:filename substitutions:
-mask S1 -sub S2
-dig n1 n2
recursion:
-r
utility:
-help | -silent | -test | -v

Example

Add .jpg to all non jpg files
perl /pathto/renner.pl . -unmask jpg$ -app .jpg

Match Criteria

$ - anchors a match to END of string
^ - anchors a match to START of string

-mask string
default: .
act only on filenames that match the string
example: perl renner.pl . -mask .txt -app .bak
append .bak to any file in currect directory matching .txt
foo.txt -> foo.txt.bak
foo.txt.bak -> foo.txt.bak.txt
example: renner.pl . -mask .txt$ -app .bak
append .bak to any file with extension .txt
foo.txt -> foo.txt.bak
foo.txt.bak -> foo.txt.bak
-unmask string
act only on filenames that do NOT match the string
example: renner.pl . -unmask ^_ -app .txt
append .txt to any file in current directory that does NOT
begin with underscore
_foo -> foo
foo -> foo.txt
NOTE: -mask and -umask may not be used in the same operation.

-dir
Rename directories in addition to plain files.
NOTE: -dir and -r (recursion) may not be used in the same operation.

Add or Remove Strings from Filename

-app string
-pre string
append or prepend string to all matching filenames
example: renner.pl . -app .bak
append .bak to all files in current directory
foo -> foo.bak
example: renner.pl . -mask .gif$ -pre _
prepend all .gif files in current directory with underscore
foo.gif -> _foo.gif
foo.txt -> foo.txt

-unapp integer
-unpre integer
delete integer characters from start or end of filename.
example: renner.pl . -mask .txt$ -unapp 4
delete extension from all txt files in current directory
foo.txt -> foo
example: renner.pl . -unmask . -unapp 1
delete leading character from all files in current directory with
no extension
afoo -> foo

Perform Substitutions on Filename

-mask searchstring - sub replacestring
replace searchstring in filename with replacestring. searchstring
is required in this form.
example: renner.pl -mask display -sub d
display1.jpg -> d1.jpg

-dig integer1 integer2
zeropad strings of integer1 consecutive digits to integer2 digits
example: renner.pl -mask .txt$ -dig 2 3
23.txt -> 023.txt
123.txt -> 123.txt
To include a space in a mask, sub, or unmask, use $space

Recursion

-r
recursively traverse directories.
NOTE: -r and -dir (rename directories) may not be used in the same operation.

Utility

-help
print this message

-silent
suppress messages

-test
describe action but don’t actually rename files

-v
print detailed messages

flagging “dictionary attack” spam at the MTA level

27 July 2006

I have a server that hosts several domain names. It gets gobs of “dictionary attack” spam to addresses that have never been valid.

Spamassassin is running at the MTA level, and set to quarantine mail between a set of threshhold values. (Mostly to give me a chance to identify false positives while I’m tuning the rulesets.) The problem is, the quarantine operation happens before the MTA decides whether to accept the incoming mail message, so mail to boty72fg@mydomain.example.com can be quarantined to my review queue even though it would normally be rejected.

I started by playing around with the blacklist_to and whitelist_to directives. The problem is, I don’t want to just whitelist_to every valid alias (because valid aliases get plenty of spam). And I don’t want to have to blacklist_to everything of the form boty72fg@mydomain.example.com. What I want is to flag all mail that doesn’t match a known alias as probable spam.

Here’s my first cut at implementing this as a meta rule:

header __EXAMPLE_ALIAS_RECOGNIZED ToCc =~ /\b(?:bogus|bugos|gosub|gubos|sobug|sugob)@/i

header __EXAMPLE_DOM_RECOGNIZED ToCc =~ /\bexample\.com/i

describe EXAMPLE_ALIAS_UNRECOGNIZED Mail to invalid alias
meta EXAMPLE_ALIAS_UNRECOGNIZED (!__EXAMPLE_ALIAS_RECOGNIZED && __EXAMPLE_DOM_RECOGNIZED)
score EXAMPLE_ALIAS_UNRECOGNIZED 5.0

Seems to work okay, but it’s a little verbose, and will need to be updated whenever valid email addresses are added or removed.

I also thought about trying something like this:

# ALL addresses @mydomain.example.com are blacklisted
score USER_IN_BLACKLIST_TO 5.0
# some addresses @mydomain.example.com are ALSO whitelisted
# negating the effect of the blacklist
score USER_IN_WHITELIST_TO -5.0
blacklist_to *mydomain.example.com
whitelist_to bogus@mydomain.example.com
whitelist_to bugos@mydomain.example.com
whitelist_to gosub@mydomain.example.com
whitelist_to gubos@mydomain.example.com
whitelist_to sobug@mydomain.example.com
whitelist_to subog@mydomain.example.com

But I’m not sure if that’s truly equivalent (?). And it’s even more verbose, and doesn’t strike me as any easier to maintain — I still have a requirement to either generate the rulesets from virtusertable or manually update the rulesets when virtusertable is updated.

Anybody got a better way to do this?

using guardMailto

14 July 2006

Introduction

guardMailto is a lightweight unobtrusive JavaScript library to make it more difficult for spammers to extract email addresses from web sites.

You don’t need to be a JavaScript expert to use it. On the other hand, it has some options intended for use by people who are comfortable programming JavaScript. My hope is that you’ll find it useful if you’ve never used JavaScript in a web page before, and even more useful if you’re a guru.

(This is the short version. introducing guardMailto is an article that discusses the design approach behind this library in more detail.)

Adding guardMailto to Your Site

You can download either the raw javascript guardMailto.js or a gzip archive guardMailto.js.gz.

Just include guardMailto.js in the <head> section of your pages:

<script type="text/javascript" src="guardMailto.js></script>

There is one function, mwmw.guardMailto(). It has 3 required arguments (and some options).

  1. “linkElement” can be either the id of the element (as a string) or an HTML Element Node
  2. “userName” is the part of the email address preceding the @ character:
    test@example.com
  3. “domain” is the part of the email address between the @ character and the top level domain (.com, .org, .net, etc.):
    test@example.com

You need a way to identify each place where you want an email link to be inserted. One quick way is to put the link text in a <span> with its “id” attribute set, like this:

<p><span id="contactMe">email me<span> for more information.</p>

(Many WYSIWYG editors create id attributes for you automatically — there’s usually a “source code” or “detail” view that will let you check. I prefer human-readable id values like “contactMe,” but id=”span018″ or id=”g78956b” will work fine too. It doesn’t matter what the id is, as long as it’s unique within the page.)

Then call mwmw.guardMailto to make the link like this:

<script type="text/javascript">
new mwmw.guardMailto("contactMe","test","example");
</script>

This will link the text “email me” from the example above to the email address test@example.com.

Remember, this has to happen after the browser displays the original span. The quick and dirty way is to put the script in the page body after the original span. That’s how this site works — the email text in the footer of this page is immediately followed by JavaScript that makes it a link.

Want to See It in Action?

You can view source on this page (scroll all the way to the bottom; guardMailto is used in the footer). There’s also a basic sample page that shows the features, and a fancier sample page that demonstrates some of the cool things you can do if you throw prototype into the mix. The sample pages also demonstrate two different techniques for adding the links in an “onload” function, which is what I would recommend in most cases.

Options

If you want to set additional options, they should be passed as a JavaScript object literal, which looks like this:

{propertyName:propertyValue, AnotherPropertyName:anotherPropertyValue}

You can see all of these options in use on the basic sample page.

The options are (in alphabetical order):

  • domainSuffix: Set your top level domain if it isn’t “com” (e.g., “edu” or “org”). Don’t include the “.”

    new mwmw.guardMailto("contactMe", "test", "example", {domainSuffix:"org"});

    To make a link to test@example.org

  • linkAddress: Replace the original link text with the email address itself. To replace “test -at- example -dot- com” with “test@example.com”:

    new mwmw.guardMailto("contactMe", "test", "example", {linkAddress:true});

  • linkClass: Specify the HTML class attribute of the new link, so you can style it with CSS:

    new mwmw.guardMailto("contactMe", "test", "example", {linkClass:"emailLink"});

    If you want multiple styles, separate them with spaces:

    new mwmw.guardMailto("contactMe", "test", "example", {linkClass:"emailLink siteNavigation"});

  • linkId: Specify the HTML id attribute of the new link, so you can style it with CSS:

    new mwmw.guardMailto("contactMe", "test", "example", {linkId:"authorContact"});

  • linkHtml: Completely replace the original link. If the browser doesn’t support replacing the contents of an element, the original contents will be left unchanged:

    new mwmw.guardMailto("contactMe", "test", "example", {linkHtml:"<img class='icon' src='/emailicon' alt='click to email' / >"});

  • linkText: Replace the text of the original link. If the browser doesn’t support replacing the text of an element, the original text will be left unchanged:

    new mwmw.guardMailto("contactMe", "test", "example", {linkText:"email the author"});

    (If you want the link text to be the email address itself, use the linkAddress option instead.)

  • linkTitle: Specify the HTML title attribute of the new link. This is often displayed as a tooltip when the user hovers over the link:

    new mwmw.guardMailto("contactMe", "test", "example", {linkTitle:"Click to email the author"});

You can mix and match almost all of these:

new mwmw.guardMailto("contactMe", "test", "example", {linkClass:"deadAuthor european" linkText:"Franz Kafka" linkTitle:"Click to email the author"});

The options linkAddress, linkHtml, and linkText are exceptions to the mix and match rule. Since they all change the contents of the link that’s displayed, you should only use one of them at a time.

License

guardMailto is released under The MIT License.

Browsers:

guardMailto has been tested in recent versions of Firefox, Internet Explorer (5.5, 6, and 7 beta), Mozilla, Netscape, and Opera, on Windows 98SE, 2000, and XP.

Current Version

1.0.01, last updated 8.jul.2006.
View changelog.

Acknowledgments

Rich Thornett provided excellent suggestions on everything from nomenclature to tweaking the API. I’m grateful for his generosity with his time and expertise. Thanks are also due Kathy Cashel for suggestions on clarifying this documentation.

Feedback?

Find it useful? Find it useless? Bugs? Tested on a different browser/platform? Suggestions? I’d love to hear from you. Leave a comment, or email me using the link in the footer.

guardMailto update

8 July 2006

After implementing guardMailto on a few web sites, I realized I hadn’t done a good job of handling cases in which the link text should be the email address itself, e.g., test@example.com.

This:

new mwmw.guardMailto("contactMe", "test", "example", {linkText:"test@example.com"});

is not a good solution because it leaves a valid email address in the page body where spammers can find it. Oops.

If you’re comfortable writing JavaScript, you might realize that the value of linkText can be any expression that evaluates to a string. So you could hide the address by doing something like this:

new mwmw.guardMailto("contactMe", "test", "example", {linkText:("test"+String.fromCharCode(64)+"example."+"com")});

But that’s clunky, redundant, and less than user-friendly for web designers who don’t care to be JavaScript gurus. The obvious thing to do was to add an option to automatically replace the original link text with the address itself, so that’s why I did for version 1.0.01. Just set option linkAddress to true like this:

new mwmw.guardMailto("contactMe", "test", "example", {linkAddress:true});

On a related note, guardMailto is perhaps a little less secure than my original “munge” library in that the default implementation does include a user name and domain string in plain text in the page. The fancier sample page has been updated to demonstrate a technique that bypasses this limitation.

introducing guardMailto

29 June 2006

The Problem

If you put something like this in your website:

<a href="mailto:test@example.com">email me!</a>

spammers can identify that test@example.com is an email address with automated tools that review thousands of websites (analogous to the “spiders” used by search engines like Google and Yahoo). They can add your address to their databases. And eventually, they will surely do so. In fact, it turns out that spammers are pretty good at extracting anything that looks like an email address, whether it’s in a mailto: link or not.

First Generation Solutions

There are already several libraries (and some commercial products) that use JavaScript to dynamically create mailto: links. I even wrote one myself a couple of years ago when I was providing web hosting for a number of small businesses.

The underlying theory is that while spammers could easily get your email address by going to the site, loading the page, and looking at the results, very few spammers will invest the time necessary to do so. If the automated tool doesn’t return an email address, they won’t check each site by hand. The economics of spamming are contingent on the incremental cost of sending each email being very, very low — anything that requires the spammer to invest time eats into the profit margin.

Most of these products used the principal of graceful degradation: If JavaScript was enabled in your browser, you got a functional email link. If JavaScript wasn’t available, you got alternate content provided in a <noscript> tag, if the page author provided it — or maybe nothing at all. Code for this approach — including my “munge” library — generally followed the following format:

A JavaScript library would be included in the <head> section of the document:

<script type="text/javascript" src="emailLib.js></script>

(Don’t worry if you’re not familiar with reading and writing JavaScript code. You don’t need to be a JavaScript expert to use guardMailto, and you can safely skip these examples.)

Then, in the body of the page, there would be some inline JavaScript that used document.write() to insert the link into the page:

<script type="text/javascript">
document.write(makeAnEmailLink("stringEncryptingEmailAddress"));
</script>

And, if you were lucky, there’d be a <noscript> tag that would provide a fallback for users without JavaScript:

<noscript>
email: test (at) example (dot) com
</noscript>

Towards a Second Generation Solution

I observed several serious problems with my “munge” library:

  • It was too complicated and hard to use. Programmers seemed comfortable with it, but web designers often weren’t, and “ordinary” people struggled mightily with it.
  • It was inflexible. It worked ok for text links that were all by themselves, but it didn’t readily support linking an image, for example.
  • It didn’t play particularly well with many “what-you-see-is-what-you-get” (WYSIWIG) web page editors.
  • It put the burden of creating the fallback <noscript> support on the page author. Many page authors were confused about how to do this effectively, and many just didn’t bother.

In the intervening time, I’ve learned a lot about object-oriented JavaScript, and I’ve become an advocate of progressive enhancement as an alternative to graceful degradation. (unobtrusive JavaScript is a related buzzword; I prefer “progressive enhancement” because it’s a design philosophy that’s applicable to more than just JavaScript.)

Progressive enhancement reverses the paradigm of graceful degradation: start with a basic html page that works “as-is,” and then enhance the functionality of the basic page for browsers that support it. Progressive enhancement makes it impossible to strand users of older browsers by omitting a <noscript> tag. It also better supports users who have a modern browser but may have JavaScript disabled or restricted for reasons of security and/or corporate policy. Finally, it often better addresses the accessibility needs of users with visual or other impairments.

Conceptually, the progressive enhancement model looks like this.

We still start with a library included in the <head> section:

<script type="text/javascript" src="emailLib.js></script>

In the body of the page, we have the link information in a format that is easily human-readable, but less easily machine-readable:

email me at <span id="contactAddress">test (at) example (dot) com</span>

And then we have some JavaScript that turns the human-readable text into a clickable link:

<script type="text/javascript">
makeAnEmailLink("contactAddress","stringEncryptingEmailAddress")
</script>

That JavaScript code can be inline in the page body, but it can also be somewhere else, so we can separate the page content from the presentation logic.

There’s one significant catch: You can’t turn the text into a link until after the browser has displayed it in the page. That makes this code a good candidate for an “onload” event, which the browser will run after the page finishes loading.

Enough Talk, Where’s the Code?

You can download either the raw javascript guardMailto.js or a gzip archive guardMailto.js.gz.

guardMailto is released under The MIT License.

Instructions for adding guardMailto to your site are covered in using guardMailto.

passing false when the default is true

24 June 2006

Lately, inspired by the prototype and scriptaculous libraries, I’ve frequently been defining JavaScript functions that accept an anonymous options object, so they can be called like this:

mynamespace.myFunction(requiredParam1, requireParam2, {
overrideSomeDefault:true,
customCallback: mynamespace.myOtherFunction
});

And I’ve gotten fond of boolean short-circuits for parsing the options object and assigning defaults:

options.overrideSomeDefault = options.overrideSomeDefault || false;

Since everything except null, 0, “”, NaN, and undefined is “truthy” in JavaScript (see Simon Willison’s great presentation “A (Re)-Introduction to JavaScript” either as pretty slides or texty notes) this is fast, terse, and readable.

Unless for some reason you explictly need to pass one of the “falsey” values. I hit this first trying to make an autocompleter library work the way I wanted it to, something along these lines:

autocompleter.init(myTarget,myCallback,{minCharsToMatch:0});
...
autocompleter {
init:function(_target,_callback,options) {
options = options || {};
options.minCharsToMatch = options.minCharsToMatch || 1;
...

Oops, I was stuck with minCharsToMatch of 1, which is not what I wanted.

I also think there are times when it makes more sense for a property to have a default value of true than false. It’s clearer (imo) for “enabled” to default true than for “disabled” to default false.

Here’s my attempt at a solution. If it’s reasonable to expect a “falsey” option value, you can do this:

option.aBool = ("undefined" != typeof option.aBool ? option.someBool : true);

Not as pretty, and not as fast. But should get the job done.