SecondHand C0D1NG

Docs first:
https://docs.python.org/3/library/glob.html

1
2

import glob
glob.glob('./[0-9].*') # ['./1.gif', './2.txt']

The glob module finds all the pathnames matching a specified pattern according to the rules used by the Unix shell, although results are returned in arbitrary order.

glob.glob(pathname, *, recursive=False)

Return a possibly-empty list of path names that match pathname, which must be a string containing a path specification. pathname can be either absolute (like /usr/src/Python-1.5/Makefile) or relative (like ../../Tools//.gif), and can contain shell-style wildcards. Broken symlinks are included in the results (as in the shell).

If recursive is true, the pattern “**” will match any files and zero or more directories and subdirectories. If the pattern is followed by an os.sep, only directories and subdirectories match.

The topic today is datetime which makes me think of life. Maybe, music is a mataphor of life.

1
2
3
4
5
6
7

import datetime

t = datetime.date(2017, 4, 4)
t.isoweekday() # 2
t.isoformat() # '2017-04-04'

one_day = datetime.timedelta(days=1)

Other things useless (maybe not) you could find on
https://docs.python.org/3/library/datetime.html

1
2
3
4
5
6
7
8
9
10
11
12

import time
from datetime import date

today = date.today()
today == date.fromtimestamp(time.time()) # True
my_birthday = date(today.year, 6, 24) # Fake birthday

if my_birthday < today:
    my_birthday = my_birthday.replace(year=today.year + 1)

time_to_birthday = abs(my_birthday - today) # why we need abs that makes previous IF statement meaningless
time_to_birthday.days

You could always find the manual. That’s enough for today.

1

pip3 install selenium

In macOS, if you have homebrew installed

1

brew install chromedriver

Then you could use Chrome in selenium like a charm.

1
2
3
4
5
6
7
8

from selenium import webdriver

url = 'https://baidu.com'
webdriver.Chrome().get(url)

# or you could
browser = webdriver.Firefox()
browser.get(url)

One example on the documentation
https://pypi.python.org/pypi/selenium

1
2
3
4
5
6
7
8
9
10
11

from selenium import webdriver
from selenium.webdriver.common.keys import Keys

browser = webdriver.Firefox() # open a new Firefox browser
browser.get('http://www.yahoo.com') # load the Yahoo homepage
assert 'Yahoo!' in browser.title # test, I suppose

elem = browser.find_element_by_name('p')  # Find the search box
elem.send_keys('seleniumhq' + Keys.RETURN) # search for “seleniumhq”

browser.quit() # close the browser

Selenium WebDriver is often used as a basis for testing web applications. Here is a simple example uisng Python’s standard unittest library:

1
2
3
4
5
6
7
8
9
10
11
12
13
14

import unittest

class GoogleTestCase(unittest.TestCase):

    def setUp(self):
        self.browser = webdriver.Firefox()
        self.addCleanup(self.browser.quit)

    def testPageTitle(self):
        self.browser.get('http://www.google.com')
        self.assertIn('Google', self.browser.title)

if __name__ == '__main__':
    unittest.main(verbosity=2)

One more thing: selenium.webdriver.common.keys

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66

class selenium.webdriver.common.keys.Keys

ADD = u'\ue025'
ALT = u'\ue00a'
ARROW_DOWN = u'\ue015'
ARROW_LEFT = u'\ue012'
ARROW_RIGHT = u'\ue014'
ARROW_UP = u'\ue013'
BACKSPACE = u'\ue003'
BACK_SPACE = u'\ue003'
CANCEL = u'\ue001'
CLEAR = u'\ue005'
COMMAND = u'\ue03d'
CONTROL = u'\ue009'
DECIMAL = u'\ue028'
DELETE = u'\ue017'
DIVIDE = u'\ue029'
DOWN = u'\ue015'
END = u'\ue010'
ENTER = u'\ue007'
EQUALS = u'\ue019'
ESCAPE = u'\ue00c'
F1 = u'\ue031'
F10 = u'\ue03a'
F11 = u'\ue03b'
F12 = u'\ue03c'
F2 = u'\ue032'
F3 = u'\ue033'
F4 = u'\ue034'
F5 = u'\ue035'
F6 = u'\ue036'
F7 = u'\ue037'
F8 = u'\ue038'
F9 = u'\ue039'
HELP = u'\ue002'
HOME = u'\ue011'
INSERT = u'\ue016'
LEFT = u'\ue012'
LEFT_ALT = u'\ue00a'
LEFT_CONTROL = u'\ue009'
LEFT_SHIFT = u'\ue008'
META = u'\ue03d'
MULTIPLY = u'\ue024'
NULL = u'\ue000'
NUMPAD0 = u'\ue01a'
NUMPAD1 = u'\ue01b'
NUMPAD2 = u'\ue01c'
NUMPAD3 = u'\ue01d'
NUMPAD4 = u'\ue01e'
NUMPAD5 = u'\ue01f'
NUMPAD6 = u'\ue020'
NUMPAD7 = u'\ue021'
NUMPAD8 = u'\ue022'
NUMPAD9 = u'\ue023'
PAGE_DOWN = u'\ue00f'
PAGE_UP = u'\ue00e'
PAUSE = u'\ue00b'
RETURN = u'\ue006'
RIGHT = u'\ue014'
SEMICOLON = u'\ue018'
SEPARATOR = u'\ue026'
SHIFT = u'\ue008'
SPACE = u'\ue00d'
SUBTRACT = u'\ue027'
TAB = u'\ue004'
UP = u'\ue013'

You should also have a look at WebDriver API
http://selenium-python.readthedocs.io/api.html

1
2
3
4
5
6
7

import webbrowser

url = 'https://baidu.com'
webbrowser.open(url, new=1, autoraise=True)
webbrowser.open(url, new=0, autoraise=True)
webbrowser.open(url, new=1, autoraise=False)
webbrowser.open(url, new=0, autoraise=False)

Display url using the default browser. If new is 0, the url is opened in the same browser window if possible. If new is 1, a new browser window is opened if possible. If new is 2, a new browser page (“tab”) is opened if possible. If autoraise is True, the window is raised if possible (note that under many window managers this will occur regardless of the setting of this variable).

No differences among the four as far as I tested on Mac with Safari. Maybe that’s because the default settings in Safari don’t allow such behavior. Anyway, the below function also isn’t working in terms of opening in a new window.

1

webbrowser.open_new(url)

In case you want to use Chrome

1
2

chrome_path = 'open -a /Applications/Google\ Chrome.app %s'
webbrowser.get(chrome_path).open(url)

A regular expression (or RE) specifies a set of strings that matches it; the functions in this module let you check if a particular string matches a given regular expression (or if a given regular expression matches a particular string, which comes down to the same thing). Blah blah blah.

Again, documentation first
https://docs.python.org/3/library/re.html
as always

1

import re

You cannot match a Unicode string with a byte pattern or vice-versa.

The special characters in RE
tl;ra (Too long; Read anyway) or maybe you should read the documentation directly.

‘.’
(Dot.) In the default mode, this matches any character except a newline. If the DOTALL flag has been specified, this matches any character including a newline.

‘^’
(Caret.) Matches the start of the string, and in MULTILINE mode also matches immediately after each newline.

‘$’
Matches the end of the string or just before the newline at the end of the string, and in MULTILINE mode also matches before a newline.

‘‘
Causes the resulting RE to match 0 or more repetitions of the preceding RE, as many repetitions as are possible. ab will match ‘a’, ‘ab’, or ‘a’ followed by any number of ‘b’s.

‘+’
Causes the resulting RE to match 1 or more repetitions of the preceding RE. ab+ will match ‘a’ followed by any non-zero number of ‘b’s; it will not match just ‘a’.

‘?’
Causes the resulting RE to match 0 or 1 repetitions of the preceding RE. ab? will match either ‘a’ or ‘ab’.

?, +?, ??
The ‘‘, ‘+’, and ‘?’ qualifiers are all greedy; they match as much text as possible. Sometimes this behaviour isn’t desired; if the RE <.*> is matched against b , it will match the entire string, and not just . Adding ? after the qualifier makes it perform the match in non-greedy or minimal fashion; as few characters as possible will be matched. Using the RE <.*?> will match only .

(?(id/name)yes-pattern|no-pattern)
Will try to match with yes-pattern if the group with given id or name exists, and with no-pattern if it doesn’t. no-pattern is optional and can be omitted. For example, (<)?(\w+@\w+(?:.\w+)+)(?(1)>|$) is a poor email matching pattern, which will match with ‘user@host.com‘ as well as ‘user@host.com‘, but not with ‘<user@host.com‘ nor ‘user@host.com>’.

\number
Matches the contents of the group of the same number. Groups are numbered starting from 1. For example, (.+) \1 matches ‘the the’ or ‘55 55’, but not ‘thethe’ (note the space after the group).

\A
Matches only at the start of the string.

\b
Matches the empty string, but only at the beginning or end of a word.

\B
Matches the empty string, but only when it is not at the beginning or end of a word. This means that r’py\B’ matches ‘python’, ‘py3’, ‘py2’, but not ‘py’, ‘py.’, or ‘py!’.

\d
For Unicode (str) patterns:
Matches any Unicode decimal digit (that is, any character in Unicode character category [Nd]). This includes [0-9], and also many other digit characters. If the ASCII flag is used only [0-9] is matched (but the flag affects the entire regular expression, so in such cases using an explicit [0-9] may be a better choice).
For 8-bit (bytes) patterns:
Matches any decimal digit; this is equivalent to [0-9].

\D
Matches any character which is not a Unicode decimal digit. This is the opposite of \d. If the ASCII flag is used this becomes the equivalent of [^0-9] (but the flag affects the entire regular expression, so in such cases using an explicit [^0-9] may be a better choice).

\s
Matches Unicode whitespace characters

\S
Matches any character which is not a Unicode whitespace character.

\w
Matches Unicode word characters

\W
Matches any character which is not a Unicode word character.

\Z
Matches only at the end of the string.

OK, let’s move on to Module Contents

1

re.compile(pattern, flags=0)

Compile a regular expression pattern into a regular expression object, which can be used for matching using its match() and search() methods

1
2

prog = re.compile(pattern)
result = prog.match(string)

is equivalent to

1

result = re.match(pattern, string)

but using re.compile() and saving the resulting regular expression object for reuse is more efficient when the expression will be used several times in a single program.

re.A
re.ASCII

re.DEBUG

re.I
re.IGNORECASE

re.L
re.LOCALE

re.M
re.MULTILINE

re.S
re.DOTALL

re.X
re.VERBOSE
This flag allows you to write regular expressions that look nicer and are more readable by allowing you to visually separate logical sections of the pattern and add comments. Whitespace within the pattern is ignored, except when in a character class or when preceded by an unescaped backslash. When a line contains a # that is not in a character class and is not preceded by an unescaped backslash, all characters from the leftmost such # through the end of the line are ignored.

This means that the two following regular expression objects that match a decimal number are functionally equal:

1
2
3
4
5
6

a = re.compile(r"""\d +  # the integral part
                   \.    # the decimal point
                   \d *  # some fractional digits""", re.X)
b = re.compile(r"\d+\.\d*")

re.search(pattern, string, flags=0)

Scan through string looking for the first location where the regular expression pattern produces a match, and return a corresponding match object.

1

re.match(pattern, string, flags=0)

If zero or more characters at the beginning (be careful) of string match the regular expression pattern, return a corresponding match object.

1

re.fullmatch(pattern, string, flags=0)

If the whole string matches the regular expression pattern, return a corresponding match object.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

re.split(pattern, string, maxsplit=0, flags=0)
# Split string by the occurrences of pattern.
re.split('\W+', 'Words, words, words.')
# ['Words', 'words', 'words', '']
re.split('(\W+)', 'Words, words, words.')
# ['Words', ', ', 'words', ', ', 'words', '.', '']
re.split('\W+', 'Words, words, words.', 1)
# ['Words', 'words, words.']
re.split('[a-f]+', '0a3B9', flags=re.IGNORECASE)
# ['0', '3', '9']
re.split('(\W+)', '...words, words...')
# ['', '...', 'words', ', ', 'words', '...', '']

re.findall(pattern, string, flags=0)
# Return all non-overlapping matches of pattern in string, as a list of strings.

re.finditer(pattern, string, flags=0)
# Return an iterator yielding match objects over all non-overlapping matches for the RE pattern in string.

re.sub(pattern, repl, string, count=0, flags=0)

Return the string obtained by replacing the leftmost non-overlapping occurrences of pattern in string by the replacement repl.

1
2
3
4
5
6
7
8
9
10

m = re.match(r"(\w+) (\w+)", "Isaac Newton, physicist")
m.group(0)       # The entire match 'Isaac Newton'
m.group(1)       # The first parenthesized subgroup. 'Isaac'
m.group(2)       # The second parenthesized subgroup. 'Newton'
m.group(1, 2)    # Multiple arguments give us a tuple. ('Isaac', 'Newton')

email = "tony@tiremove_thisger.net"
m = re.search("remove_this", email)
email[:m.start()] + email[m.end():]
# 'tony@tiger.net'

Regular Expression Examples in standard docs

1
2
3
4

def displaymatch(match):
    if match is None:
        return None
    return '<Match: %r, groups=%r>' % (match.group(), match.groups())

Suppose you are writing a poker program where a player’s hand is represented as a 5-character string with each character representing a card, “a” for ace, “k” for king, “q” for queen, “j” for jack, “t” for 10, and “2” through “9” representing the card with that value.

To see if a given string is a valid hand, one could do the following:

1
2
3
4
5

valid = re.compile(r"^[a2-9tjqk]{5}$")
displaymatch(valid.match("akt5q"))  # Valid. "<Match: 'akt5q', groups=()>"
displaymatch(valid.match("akt5e"))  # Invalid.
displaymatch(valid.match("akt"))    # Invalid.
displaymatch(valid.match("727ak"))  # Valid. "<Match: '727ak', groups=()>"

That last hand, “727ak”, contained a pair, or two of the same valued cards. To match this with a regular expression, one could use backreferences as such:

1
2
3
4

pair = re.compile(r".*(.).*\1")
displaymatch(pair.match("717ak"))     # Pair of 7s. "<Match: '717', groups=('7',)>"
displaymatch(pair.match("718ak"))     # No pairs.
displaymatch(pair.match("354aa"))     # Pair of aces. "<Match: '354aa', groups=('a',)>"

To find out what card the pair consists of, one could use the group() method of the match object in the following manner:

1
2
3
4
5
6
7
8
9

pair.match("717ak").group(1) # '7'
# Error because re.match() returns None, which doesn't have a group() method:
pair.match("718ak").group(1)
Traceback (most recent call last):
  File "<pyshell#23>", line 1, in <module>
    re.match(r".*(.).*\1", "718ak").group(1)
AttributeError: 'NoneType' object has no attribute 'group'

pair.match("354aa").group(1) # 'a'

search() vs. match()

of course you can look up on stackoverflow: re.match() checks for a match only at the beginning of the string, while re.search() checks for a match anywhere in the string (this is what Perl does by default).

1
2

text = "He was carefully disguised but captured quickly by police."
re.findall(r"\w+ly", text) # ['carefully', 'quickly']

Writing a Tokenizer

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45

import collections
import re

Token = collections.namedtuple('Token', ['typ', 'value', 'line', 'column'])

def tokenize(code):
    keywords = {'IF', 'THEN', 'ENDIF', 'FOR', 'NEXT', 'GOSUB', 'RETURN'}
    token_specification = [
        ('NUMBER',  r'\d+(\.\d*)?'),  # Integer or decimal number
        ('ASSIGN',  r':='),           # Assignment operator
        ('END',     r';'),            # Statement terminator
        ('ID',      r'[A-Za-z]+'),    # Identifiers
        ('OP',      r'[+\-*/]'),      # Arithmetic operators
        ('NEWLINE', r'\n'),           # Line endings
        ('SKIP',    r'[ \t]+'),       # Skip over spaces and tabs
        ('MISMATCH',r'.'),            # Any other character
    ]
    tok_regex = '|'.join('(?P<%s>%s)' % pair for pair in token_specification)
    line_num = 1
    line_start = 0
    for mo in re.finditer(tok_regex, code):
        kind = mo.lastgroup
        value = mo.group(kind)
        if kind == 'NEWLINE':
            line_start = mo.end()
            line_num += 1
        elif kind == 'SKIP':
            pass
        elif kind == 'MISMATCH':
            raise RuntimeError(f'{value!r} unexpected on line {line_num}')
        else:
            if kind == 'ID' and value in keywords:
                kind = value
            column = mo.start() - line_start
            yield Token(kind, value, line_num, column)

statements = '''
    IF quantity THEN
        total := total + price * quantity;
        tax := price * 0.05;
    ENDIF;
'''

for token in tokenize(statements):
    print(token)

The tokenizer produces the following output:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

Token(typ='IF', value='IF', line=2, column=4)
Token(typ='ID', value='quantity', line=2, column=7)
Token(typ='THEN', value='THEN', line=2, column=16)
Token(typ='ID', value='total', line=3, column=8)
Token(typ='ASSIGN', value=':=', line=3, column=14)
Token(typ='ID', value='total', line=3, column=17)
Token(typ='OP', value='+', line=3, column=23)
Token(typ='ID', value='price', line=3, column=25)
Token(typ='OP', value='*', line=3, column=31)
Token(typ='ID', value='quantity', line=3, column=33)
Token(typ='END', value=';', line=3, column=41)
Token(typ='ID', value='tax', line=4, column=8)
Token(typ='ASSIGN', value=':=', line=4, column=12)
Token(typ='ID', value='price', line=4, column=15)
Token(typ='OP', value='*', line=4, column=21)
Token(typ='NUMBER', value='0.05', line=4, column=23)
Token(typ='END', value=';', line=4, column=27)
Token(typ='ENDIF', value='ENDIF', line=5, column=4)
Token(typ='END', value=';', line=5, column=9)

I know, I know it’s a headache to read these, try PyMOTW-3 instead if you want
https://pymotw.com/3/re/index.html

Out there is an alternative regular expression module, to replace re, called regex.
https://pypi.python.org/pypi/regex/

And read this:
10 Tips to Get More out of Your Regexes
http://pybit.es/mastering-regex.html

First of all, in Python 3

1

import random

You would find the documentation on page of
https://docs.python.org/3/library/random.html
in which it said that almost all module functions depend on the basic function random(), which generates a random float uniformly in the semi-open range [0.0, 1.0).

Normally I would not talk about the deeper side of the module such as the core generator. However in this case, you have to know that Python uses the Mersenne Twister as the core generator.

It produces 53-bit precision floats and has a period of 2**19937-1. The underlying implementation in C is both fast and threadsafe. The Mersenne Twister is one of the most extensively tested random number generators in existence. However, being completely deterministic, it is not suitable for all purposes, and is completely unsuitable for cryptographic purposes.

I say it again that random Module is unsuitable for cryptographic purposes. Other than that you could use it all the time.

1
2
3
4
5
6
7
8
9
10
11
12

random.random() # 0.01658182124355423
random.uniform(1.5, 100.5) # 17.133710707896064
random.seed(1) # seeding, otherwise, the current time is used
random.randint(1, 100) # Random Integers
random.randint(-5, 5)
random.randint(a, b) # Return a random integer N such that a <= N <= b
random.randrange(0, 101, 5)
random.choice('abc')
random.choices(['red', 'black', 'green'], [18, 18, 2], k=6) # ['red', 'green', 'black', 'black', 'red', 'black']
words =['a','b','c','d']
random.shuffle(words) # ['d', 'b', 'a', 'c']
random.sample('abcdefg',2) # ['f', 'a']

By the way, for security or cryptographic uses, there is a module called secrets.

According to 16.3. time — Time access and conversions
https://docs.python.org/3/library/time.html

1

This module provides various time-related functions. 

Or you can say in this module there are

1

Functions for manipulating clock time.

as in the PYMOTW-3
https://pymotw.com/3/time/index.html

I am not going to list every function available in the ‘time’ module, only the core functions such as time(), etc.

1
2
3

import time

time.time()

You would see number of seconds since the start of the “epoch” as a floating point value, e.g. 1491220050.174642.

1
2
3
4
5
6
7
8

time.ctime() # 'Mon Apr  3 19:49:02 2017'
time.ctime(15) # 'Thu Jan  1 08:00:15 1970'
time.ctime(1) # 'Thu Jan  1 08:00:01 1970'
time.ctime(time.time()+15) # 'Mon Apr  3 19:52:10 2017'

start = time.monotonic()
time.sleep(15.5) # sleep for at least 15.5 seconds, see PEP 475
end = time.monotonic()

Only the difference between the results of consecutive calls is valid. The same applies to time.perf_counter() and time.process_time()

1

time.gmtime()

time.struct_time(tm_year=2017, tm_mon=4, tm_mday=3, tm_hour=12, tm_min=3, tm_sec=33, tm_wday=0, tm_yday=93, tm_isdst=0)

1
2
3
4

time.gmtime().tm_year # 2017
time.gmtime()[0] # 2017

time.perf_counter()

Return the value (in fractional seconds) of a performance counter, i.e. a clock with the highest available resolution to measure a short duration. It does include time elapsed during sleep and is system-wide.

1

time.process_time()

Return the value (in fractional seconds) of the sum of the system and user CPU time of the current process. It does not include time elapsed during sleep. It is process-wide by definition.

The above should be enough for now.

Life is short, I use Python.

>>> import this

The Zen of Python, by Tim Peters

Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!```