1.1 --- /dev/null Thu Jan 01 00:00:00 1970 +0000
1.2 +++ b/OpenSecurity/Readme.md Mon Dec 02 14:02:05 2013 +0100
1.3 @@ -0,0 +1,151 @@
1.4 +# How To OpenSecurity Demo
1.5 +
1.6 +## Installation
1.7 +
1.8 +1. Copy the OpenSecurity parent Folder as-is to "C:\Program Files"
1.9 +
1.10 +2. Inside this folder you find:
1.11 +
1.12 + OpenSecurity/
1.13 + ├── client ............... OpenSecurity client code
1.14 + ├── cygwin ............... A cygwin subsystem used inside OpenSecurity
1.15 + ├── gfx .................. OpenSecurity images and icons used
1.16 + ├── install .............. Necessary 3rd party installment
1.17 + └── server ............... OpenSecuirty server code
1.18 +
1.19 +3. Switch into the "install" folder. There you have:
1.20 +
1.21 + OpenSecurity/
1.22 + └── install/
1.23 + ├── OpenSecurity.reg
1.24 + ├── PyQt4-4.10.3-gpl-Py2.7-Qt4.8.5-x32.exe
1.25 + ├── PyQt4-4.10.3-gpl-Py2.7-Qt4.8.5-x64.exe
1.26 + ├── python-2.7.6.amd64.msi
1.27 + ├── python-2.7.6.msi
1.28 + ├── VirtualBox-4.3.4-91027-Win.exe
1.29 + └── web.py-0.37
1.30 +
1.31 + Please install the software via double-click:
1.32 +
1.33 + * python-2.7.6.msi on 32-Bit
1.34 + _or_
1.35 + python-2.7.6.amd64.msi on 64-Bit
1.36 +
1.37 + * PyQt4-4.10.3-gpl-Py2.7-Qt4.8.5-x32.exe on 32-Bit
1.38 + _or_
1.39 + PyQt4-4.10.3-gpl-Py2.7-Qt4.8.5-x64.exe on 64-Bit
1.40 +
1.41 + * VirtualBox-4.3.4-91027-Win.exe
1.42 +
1.43 + If you didn't change any settings you'll have a Win32 Python
1.44 + installation at C:\Python27 right now. This is important for
1.45 + the next step.
1.46 +
1.47 + 1. Open up a cmd box --> Start / Execute / "cmd"
1.48 + 2. Switch into the OpenSecurity folder where the web.py resides:
1.49 +
1.50 + C:> cd C:\Program Files\OpenSecurity\install\web.py-0.37
1.51 +
1.52 + 3. Install web.py by calling the setup.py with the "install" command
1.53 + from within a python shell:
1.54 +
1.55 + C:\Program Files\OpenSecurity\install\web.py-0.37> C:\Python27\python.exe setup.py install
1.56 + running install
1.57 + running build
1.58 + running build_py
1.59 + creating build
1.60 + creating build\lib
1.61 + creating build\lib\web
1.62 + copying web\application.py -> build\lib\web
1.63 + copying web\browser.py -> build\lib\web
1.64 + copying web\db.py -> build\lib\web
1.65 + copying web\debugerror.py -> build\lib\web
1.66 + copying web\form.py -> build\lib\web
1.67 + ...
1.68 +
1.69 +4. Finally update your registry by double-clicking the OpenSecurity.reg file.
1.70 +
1.71 +5. To make changes in effect (automatically starting the OpenSecurity client and server daemons) you should restrat the machine.
1.72 +
1.73 +NOTE:
1.74 + Windows will pop up an UAC dialog for X11, OpenSecurity Client Daemon and OpenSecuirty Server Daemon
1.75 +
1.76 +
1.77 +## VirtualBox VM Images
1.78 +
1.79 +For the current setup to work you need at least a single Virtual Machine:
1.80 +
1.81 +1. Create a Virtual Machine for Debian Linux
1.82 + --> The machine should be named 'Debian 7'
1.83 + --> There must be a user called 'user'
1.84 +
1.85 +2. Have a Debian 7 (or 7.2) network installation ready and install a fresh new Debian system, with a user called 'user'.
1.86 +
1.87 +3. Create 2 (!) Network Interfaces for your Virtual Machine
1.88 + a) The first will be set to "NAT" --> this will be eth0
1.89 + b) The second will be set to "Host-Only Adapter" --> this will be eth1
1.90 +
1.91 +4. Power up the Virtual Machine and set the network interface configuration (/etc/network/interfaces) to:
1.92 +
1.93 + auto lo
1.94 + iface lo
1.95 +
1.96 + auto eth0
1.97 + allow-hotplug eth0
1.98 + iface eth0 inet dhcp
1.99 +
1.100 + auto eth1
1.101 + iface eth1 inet static
1.102 + address 192.168.56.101
1.103 + netmask 255.255.255.0
1.104 + gateway 192.168.56.1
1.105 +
1.106 +5. Create a passwordless SSH connection from within Cygwin into the VM:
1.107 +
1.108 + a) ensure the VM is started and you have a user login called 'user'.
1.109 + b) start a cygwin shell by double-clicking "C:\Program Files\OpenSecurity\cygwin\Cygwin.vbs"
1.110 + c) generate a ssh-key
1.111 +
1.112 + $ ssh-keygen
1.113 +
1.114 + --> do not set passphrases, leave all to default
1.115 + d) copy the public key to the virtual machine
1.116 +
1.117 + $ scp ~/.ssh/id_rsa.pub user@192.168.56.101:
1.118 +
1.119 + e) add the public key to the list of authorized keys:
1.120 +
1.121 + - login into the virtual machine
1.122 + - open up a terminal
1.123 +
1.124 + $ mkdir ~/.ssh &> /dev/null
1.125 + $ cat id_rsa.pub >> ~/.ssh/authorized_keys
1.126 +
1.127 + f) test the passwordless connection by open the cyginw command prompt on the Windows Host again:
1.128 +
1.129 + $ ssh user@192.168.56.101
1.130 +
1.131 + --> this should now give you a login shell on the virtual machine without a password request.
1.132 +
1.133 + (you can now safely delete the id_rsa.pub file in your virtual machine's home)
1.134 +
1.135 +
1.136 +## Demonstration
1.137 +
1.138 +* Start the Virtual Machine
1.139 + --> You do not have to log in. Just start the machine. If the X11-Login Screen appears, all is done.
1.140 +
1.141 +* Start the opensecurity-client by calling
1.142 +
1.143 + NOTE: you may omit this step if you double-clicked the OpenSecuirty.reg file previously.
1.144 +
1.145 + C:> C:\
1.146 + C:> cd "C:\Program Files\OpenSecurity\client"
1.147 + C:\Program Files\OpenSecurity\client> start "opensecurity_client_restful_server.py 8090"
1.148 +
1.149 +
1.150 +* Open Up a browser and type:
1.151 +
1.152 + "http://127.0.0.1:8090"
1.153 +
1.154 + HAVE FUN! =D
2.1 --- /dev/null Thu Jan 01 00:00:00 1970 +0000
2.2 +++ b/OpenSecurity/client/about.py Mon Dec 02 14:02:05 2013 +0100
2.3 @@ -0,0 +1,126 @@
2.4 +#!/bin/env python
2.5 +# -*- coding: utf-8 -*-
2.6 +
2.7 +# ------------------------------------------------------------
2.8 +# about-dialog
2.9 +#
2.10 +# tell the user about the project
2.11 +#
2.12 +# Autor: Oliver Maurhart, <oliver.maurhart@ait.ac.at>
2.13 +#
2.14 +# Copyright (C) 2013 AIT Austrian Institute of Technology
2.15 +# AIT Austrian Institute of Technology GmbH
2.16 +# Donau-City-Strasse 1 | 1220 Vienna | Austria
2.17 +# http://www.ait.ac.at
2.18 +#
2.19 +# This program is free software; you can redistribute it and/or
2.20 +# modify it under the terms of the GNU General Public License
2.21 +# as published by the Free Software Foundation version 2.
2.22 +#
2.23 +# This program is distributed in the hope that it will be useful,
2.24 +# but WITHOUT ANY WARRANTY; without even the implied warranty of
2.25 +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
2.26 +# GNU General Public License for more details.
2.27 +#
2.28 +# You should have received a copy of the GNU General Public License
2.29 +# along with this program; if not, write to the Free Software
2.30 +# Foundation, Inc., 51 Franklin Street, Fifth Floor,
2.31 +# Boston, MA 02110-1301, USA.
2.32 +# ------------------------------------------------------------
2.33 +
2.34 +
2.35 +# ------------------------------------------------------------
2.36 +# imports
2.37 +
2.38 +import os
2.39 +
2.40 +from PyQt4 import QtCore
2.41 +from PyQt4 import QtGui
2.42 +
2.43 +# local
2.44 +from environment import Environment
2.45 +
2.46 +# ------------------------------------------------------------
2.47 +# vars
2.48 +
2.49 +
2.50 +ABOUT_TEXT = """
2.51 +<html>
2.52 +<body bgcolor="#FFFFFF">
2.53 +
2.54 +<div align="center">
2.55 +<p/>
2.56 +<img src="image:ait_logo_no_claim.png"/>
2.57 +<p/>
2.58 +<h1>OpenSecurity</h1>
2.59 +<p/>
2.60 +</div>
2.61 +<p/>
2.62 +Blah ...<br/>
2.63 +
2.64 +<p>
2.65 +Copyright (C) 2013, AIT Austrian Institute of Technology<br/>
2.66 +AIT Austrian Institute of Technology GmbH<br/>
2.67 +Donau-City-Strasse 1 | 1220 Vienna | Austria<br/>
2.68 +<a href="http://www.ait.ac.at">http://www.ait.ac.at</a>
2.69 +</p>
2.70 +</div>
2.71 +
2.72 +</body>
2.73 +</html>
2.74 +""";
2.75 +
2.76 +
2.77 +# ------------------------------------------------------------
2.78 +# code
2.79 +
2.80 +
2.81 +class About(QtGui.QDialog):
2.82 +
2.83 + """Show some about stuff."""
2.84 +
2.85 + def __init__(self, parent = None, flags = QtCore.Qt.WindowFlags(0)):
2.86 +
2.87 + # super call and widget init
2.88 + super(About, self).__init__(parent, flags)
2.89 +
2.90 + # setup image search path
2.91 + QtCore.QDir.setSearchPaths("image", QtCore.QStringList(os.path.join(Environment('opensecurity').data_path, '..', 'gfx')));
2.92 +
2.93 + self.setWindowTitle('About OpenSecuirty ...')
2.94 + self.setup_ui()
2.95 +
2.96 +
2.97 + def setup_ui(self):
2.98 +
2.99 + """Create the widgets."""
2.100 +
2.101 + lyMain = QtGui.QVBoxLayout(self)
2.102 + lyMain.setContentsMargins(8, 8, 8, 8)
2.103 +
2.104 + lbAbout = QtGui.QLabel()
2.105 + lbAbout.setStyleSheet("QWidget { background: white; color: black; };")
2.106 + lbAbout.setText(ABOUT_TEXT)
2.107 + lbAbout.setContentsMargins(12, 12, 12, 12)
2.108 +
2.109 + scAbout = QtGui.QScrollArea()
2.110 + scAbout.setWidget(lbAbout)
2.111 + scAbout.viewport().setStyleSheet("QWidget { background: white; color: black; };")
2.112 + lyMain.addWidget(scAbout)
2.113 +
2.114 + # buttons
2.115 + lyButton = QtGui.QHBoxLayout()
2.116 + lyMain.addLayout(lyButton)
2.117 +
2.118 + lyButton.addStretch(1)
2.119 + btnOk = QtGui.QPushButton('&Ok', self)
2.120 + btnOk.setMinimumWidth(100)
2.121 + lyButton.addWidget(btnOk)
2.122 +
2.123 + # connectors
2.124 + btnOk.clicked.connect(self.accept)
2.125 +
2.126 + # reduce to the max
2.127 + self.setMinimumSize(400, 200)
2.128 + self.resize(lyMain.minimumSize())
2.129 +
3.1 --- /dev/null Thu Jan 01 00:00:00 1970 +0000
3.2 +++ b/OpenSecurity/client/credentials.py Mon Dec 02 14:02:05 2013 +0100
3.3 @@ -0,0 +1,160 @@
3.4 +#!/bin/env python
3.5 +# -*- coding: utf-8 -*-
3.6 +
3.7 +# ------------------------------------------------------------
3.8 +# credentials-dialog
3.9 +#
3.10 +# ask the user credentials
3.11 +#
3.12 +# Autor: Oliver Maurhart, <oliver.maurhart@ait.ac.at>
3.13 +#
3.14 +# Copyright (C) 2013 AIT Austrian Institute of Technology
3.15 +# AIT Austrian Institute of Technology GmbH
3.16 +# Donau-City-Strasse 1 | 1220 Vienna | Austria
3.17 +# http://www.ait.ac.at
3.18 +#
3.19 +# This program is free software; you can redistribute it and/or
3.20 +# modify it under the terms of the GNU General Public License
3.21 +# as published by the Free Software Foundation version 2.
3.22 +#
3.23 +# This program is distributed in the hope that it will be useful,
3.24 +# but WITHOUT ANY WARRANTY; without even the implied warranty of
3.25 +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
3.26 +# GNU General Public License for more details.
3.27 +#
3.28 +# You should have received a copy of the GNU General Public License
3.29 +# along with this program; if not, write to the Free Software
3.30 +# Foundation, Inc., 51 Franklin Street, Fifth Floor,
3.31 +# Boston, MA 02110-1301, USA.
3.32 +# ------------------------------------------------------------
3.33 +
3.34 +
3.35 +# ------------------------------------------------------------
3.36 +# imports
3.37 +
3.38 +import sys
3.39 +
3.40 +from PyQt4 import QtCore
3.41 +from PyQt4 import QtGui
3.42 +
3.43 +# local
3.44 +from about import About
3.45 +
3.46 +# ------------------------------------------------------------
3.47 +# code
3.48 +
3.49 +
3.50 +class Credentials(QtGui.QDialog):
3.51 +
3.52 + """Ask the user for credentials."""
3.53 +
3.54 + def __init__(self, text, parent = None, flags = QtCore.Qt.WindowFlags(0)):
3.55 +
3.56 + super(Credentials, self).__init__(parent, flags)
3.57 + self.setWindowTitle('OpenSecuirty Credentials Request')
3.58 + self.setup_ui()
3.59 +
3.60 + # positionate ourself central
3.61 + screen = QtGui.QDesktopWidget().screenGeometry()
3.62 + self.resize(self.geometry().width() * 1.25, self.geometry().height())
3.63 + size = self.geometry()
3.64 + self.move((screen.width() - size.width()) / 2, (screen.height() - size.height()) / 2)
3.65 +
3.66 + # fix up text
3.67 + self.lbText.setText(text)
3.68 +
3.69 +
3.70 + def clicked_about(self):
3.71 + """clicked the about button"""
3.72 + dlgAbout = About()
3.73 + dlgAbout.exec_()
3.74 +
3.75 +
3.76 + def clicked_cancel(self):
3.77 + """clicked the cancel button"""
3.78 + self.reject()
3.79 +
3.80 +
3.81 + def clicked_ok(self):
3.82 + """clicked the ok button"""
3.83 + sys.stdout.write('{ ')
3.84 + sys.stdout.write('\'user\': \'')
3.85 + sys.stdout.write(self.edUser.text())
3.86 + sys.stdout.write('\', ')
3.87 + sys.stdout.write('\'password\': \'')
3.88 + sys.stdout.write(self.edPassword.text())
3.89 + sys.stdout.write('\' ')
3.90 + sys.stdout.write('}\n')
3.91 + self.accept()
3.92 +
3.93 +
3.94 + def setup_ui(self):
3.95 +
3.96 + """Create the widgets."""
3.97 +
3.98 + lyMain = QtGui.QVBoxLayout(self)
3.99 + lyMain.setContentsMargins(8, 8, 8, 8)
3.100 +
3.101 + # content area: left pixmap, right text
3.102 + lyContent = QtGui.QHBoxLayout()
3.103 + lyMain.addLayout(lyContent)
3.104 +
3.105 + # pixmap
3.106 + lbPix = QtGui.QLabel()
3.107 + lbPix.setPixmap(QtGui.QPixmapCache.find('opensecurity_icon_64'))
3.108 + lyContent.addWidget(lbPix, 0, QtCore.Qt.Alignment(QtCore.Qt.AlignTop + QtCore.Qt.AlignHCenter))
3.109 + lyContent.addSpacing(16)
3.110 +
3.111 + # text ...
3.112 + lyText = QtGui.QGridLayout()
3.113 + lyContent.addLayout(lyText)
3.114 + self.lbText = QtGui.QLabel()
3.115 + lyText.addWidget(self.lbText, 0, 0, 1, 2)
3.116 +
3.117 + lbUser = QtGui.QLabel('&User:')
3.118 + lyText.addWidget(lbUser, 1, 0)
3.119 + self.edUser = QtGui.QLineEdit()
3.120 + lyText.addWidget(self.edUser, 1, 1)
3.121 + lbUser.setBuddy(self.edUser)
3.122 +
3.123 + lbPassword = QtGui.QLabel('&Password:')
3.124 + lyText.addWidget(lbPassword, 2, 0)
3.125 + self.edPassword = QtGui.QLineEdit()
3.126 + self.edPassword.setEchoMode(QtGui.QLineEdit.Password)
3.127 + lyText.addWidget(self.edPassword, 2, 1)
3.128 + lbPassword.setBuddy(self.edPassword)
3.129 +
3.130 + lyText.addWidget(QtGui.QWidget(), 3, 0, 1, 2)
3.131 + lyText.setColumnStretch(1, 1)
3.132 + lyText.setRowStretch(3, 1)
3.133 +
3.134 + lyMain.addStretch(1)
3.135 +
3.136 + # buttons
3.137 + lyButton = QtGui.QHBoxLayout()
3.138 + lyMain.addLayout(lyButton)
3.139 +
3.140 + lyButton.addStretch(1)
3.141 + btnOk = QtGui.QPushButton('&Ok', self)
3.142 + btnOk.setDefault(True)
3.143 + btnOk.setMinimumWidth(100)
3.144 + lyButton.addWidget(btnOk)
3.145 + btnCancel = QtGui.QPushButton('&Cancel', self)
3.146 + btnCancel.setMinimumWidth(100)
3.147 + lyButton.addWidget(btnCancel)
3.148 + btnAbout = QtGui.QPushButton('&About', self)
3.149 + btnAbout.setMinimumWidth(100)
3.150 + lyButton.addWidget(btnAbout)
3.151 +
3.152 + button_width = max(btnOk.width(), btnCancel.width(), btnAbout.width())
3.153 + btnOk.setMinimumWidth(button_width)
3.154 + btnCancel.setMinimumWidth(button_width)
3.155 + btnAbout.setMinimumWidth(button_width)
3.156 +
3.157 + # reduce to the max
3.158 + self.resize(lyMain.minimumSize())
3.159 +
3.160 + # connectors
3.161 + btnOk.clicked.connect(self.clicked_ok)
3.162 + btnCancel.clicked.connect(self.clicked_cancel)
3.163 + btnAbout.clicked.connect(self.clicked_about)
4.1 --- /dev/null Thu Jan 01 00:00:00 1970 +0000
4.2 +++ b/OpenSecurity/client/cygwin.py Mon Dec 02 14:02:05 2013 +0100
4.3 @@ -0,0 +1,105 @@
4.4 +#!/bin/env python
4.5 +# -*- coding: utf-8 -*-
4.6 +
4.7 +# ------------------------------------------------------------
4.8 +# cygwin command
4.9 +#
4.10 +# executes a cygwin command inside the opensecurity project
4.11 +#
4.12 +# Autor: Oliver Maurhart, <oliver.maurhart@ait.ac.at>
4.13 +#
4.14 +# Copyright (C) 2013 AIT Austrian Institute of Technology
4.15 +# AIT Austrian Institute of Technology GmbH
4.16 +# Donau-City-Strasse 1 | 1220 Vienna | Austria
4.17 +# http://www.ait.ac.at
4.18 +#
4.19 +# This program is free software; you can redistribute it and/or
4.20 +# modify it under the terms of the GNU General Public License
4.21 +# as published by the Free Software Foundation version 2.
4.22 +#
4.23 +# This program is distributed in the hope that it will be useful,
4.24 +# but WITHOUT ANY WARRANTY; without even the implied warranty of
4.25 +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
4.26 +# GNU General Public License for more details.
4.27 +#
4.28 +# You should have received a copy of the GNU General Public License
4.29 +# along with this program; if not, write to the Free Software
4.30 +# Foundation, Inc., 51 Franklin Street, Fifth Floor,
4.31 +# Boston, MA 02110-1301, USA.
4.32 +# ------------------------------------------------------------
4.33 +
4.34 +
4.35 +# ------------------------------------------------------------
4.36 +# imports
4.37 +
4.38 +import os
4.39 +import subprocess
4.40 +import sys
4.41 +
4.42 +# local
4.43 +from environment import Environment
4.44 +
4.45 +
4.46 +# ------------------------------------------------------------
4.47 +# code
4.48 +
4.49 +
4.50 +class Cygwin(object):
4.51 +
4.52 + """Some nifty methods working with Cygwin"""
4.53 +
4.54 + def __call__(self, command, stdin = subprocess.PIPE, stdout = subprocess.PIPE, stderr = subprocess.PIPE):
4.55 + """make an instance of this object act as a function"""
4.56 + return self.execute(command, stdin, stdout, stderr)
4.57 +
4.58 +
4.59 + @staticmethod
4.60 + def root():
4.61 + """get the path to our local cygwin installment"""
4.62 + return os.path.join(Environment('OpenSecurity').prefix_path, '..', 'cygwin')
4.63 +
4.64 +
4.65 + def execute(self, command, stdin = subprocess.PIPE, stdout = subprocess.PIPE, stderr = subprocess.PIPE):
4.66 + """execute a cygwin shell command
4.67 +
4.68 + command is list of arguments like ['/bin/ls', '-al', '-h']
4.69 +
4.70 + a Popen object is returned"""
4.71 + command_path = Cygwin.root() + os.sep.join(command[0].split('/'))
4.72 + command = [command_path] + command[1:]
4.73 +
4.74 + return subprocess.Popen(command, shell = False, stdin = stdin, stdout = stdout, stderr = stderr)
4.75 +
4.76 +
4.77 + @staticmethod
4.78 + def is_X11_running():
4.79 + """check if we can connect to a X11 running instance"""
4.80 + p = Cygwin()(['/bin/bash', '-c', 'DISPLAY=:0 /usr/bin/xset -q'])
4.81 + stdout, stderr = p.communicate()
4.82 + return p.returncode == 0
4.83 +
4.84 +
4.85 + @staticmethod
4.86 + def start_X11():
4.87 + """start X11 in the background (if not already running) on DISPLAY=:0"""
4.88 +
4.89 + # do not start if already running
4.90 + if Cygwin.is_X11_running():
4.91 + return
4.92 +
4.93 + # launch X11 (forget output and return immediately)
4.94 + p = Cygwin()(['/bin/bash', '--login', '-i', '-c', ' X :0 -multiwindow'], stdin = None, stdout = None, stderr = None)
4.95 +
4.96 +
4.97 +# start
4.98 +if __name__ == "__main__":
4.99 +
4.100 + # execute what is given on the command line
4.101 + c = Cygwin()
4.102 + p = c(sys.argv[1:])
4.103 +
4.104 + # wait until the process finished and grab the output
4.105 + stdout, stderr = p.communicate()
4.106 + print('=== call result on stdout: ===\n' + stdout)
4.107 + print('=== call result on stderr: ===\n' + stderr)
4.108 +
5.1 --- /dev/null Thu Jan 01 00:00:00 1970 +0000
5.2 +++ b/OpenSecurity/client/environment.py Mon Dec 02 14:02:05 2013 +0100
5.3 @@ -0,0 +1,103 @@
5.4 +#!/bin/env python
5.5 +# -*- coding: utf-8 -*-
5.6 +
5.7 +# ------------------------------------------------------------
5.8 +# environment.py
5.9 +#
5.10 +# pick some current environment infos
5.11 +#
5.12 +# Autor: Oliver Maurhart, <oliver.maurhart@ait.ac.at>
5.13 +#
5.14 +# Copyright (C) 2013 AIT Austrian Institute of Technology
5.15 +# AIT Austrian Institute of Technology GmbH
5.16 +# Donau-City-Strasse 1 | 1220 Vienna | Austria
5.17 +# http://www.ait.ac.at
5.18 +#
5.19 +# This program is free software; you can redistribute it and/or
5.20 +# modify it under the terms of the GNU General Public License
5.21 +# as published by the Free Software Foundation version 2.
5.22 +#
5.23 +# This program is distributed in the hope that it will be useful,
5.24 +# but WITHOUT ANY WARRANTY; without even the implied warranty of
5.25 +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
5.26 +# GNU General Public License for more details.
5.27 +#
5.28 +# You should have received a copy of the GNU General Public License
5.29 +# along with this program; if not, write to the Free Software
5.30 +# Foundation, Inc., 51 Franklin Street, Fifth Floor,
5.31 +# Boston, MA 02110-1301, USA.
5.32 +# ------------------------------------------------------------
5.33 +
5.34 +
5.35 +# ------------------------------------------------------------
5.36 +# imports
5.37 +
5.38 +import os
5.39 +import os.path
5.40 +import sys
5.41 +
5.42 +
5.43 +# ------------------------------------------------------------
5.44 +# code
5.45 +
5.46 +
5.47 +class Environment(object):
5.48 +
5.49 + """Hold some nifty environment stuff in a dedicated class."""
5.50 +
5.51 + def __init__(self, application = None):
5.52 +
5.53 + # if we ain't got a path to start from, all is valid/lost
5.54 + if len(sys.path[0]) == 0:
5.55 + self._prefix_path = ''
5.56 + self._data_path = ''
5.57 + return
5.58 +
5.59 + # the prefix path
5.60 + #
5.61 + # - on Linux: this is ../../ to the current executable
5.62 + # e.g. "/usr/bin/myprogram" --> "/usr"
5.63 + #
5.64 + # - on Windows (inkl. Cygwin): this is the installation folder
5.65 + # e.g. "C:/Program Files/MyProgram/myprogam" --> "C:/Program Files/MyProgram"
5.66 + #
5.67 + if sys.platform == 'linux2':
5.68 + self._prefix_path = os.path.split(sys.path[0])[0]
5.69 + elif sys.platform == 'win32' or sys.platform == 'cygwin':
5.70 + self._prefix_path = sys.path[0]
5.71 +
5.72 + # the data path where all data files are stored
5.73 + if sys.platform == 'linux2':
5.74 + if not application is None:
5.75 + self._data_path = os.path.join(self._prefix_path, os.path.join('share', application))
5.76 + else:
5.77 + self._data_path = os.path.join(self._prefix_path, 'share')
5.78 + elif sys.platform == 'win32' or sys.platform == 'cygwin':
5.79 + self._data_path = self._prefix_path
5.80 +
5.81 +
5.82 + def data_path_get(self):
5.83 + """dat_path get"""
5.84 + return self._data_path
5.85 +
5.86 + data_path = property(data_path_get)
5.87 +
5.88 +
5.89 + def prefix_path_get(self):
5.90 + """prefix_path get"""
5.91 + return self._prefix_path
5.92 +
5.93 + prefix_path = property(prefix_path_get)
5.94 +
5.95 +# test method
5.96 +def test():
5.97 +
5.98 + """Test: class Environment"""
5.99 + e = Environment('My Application')
5.100 + print('prefix_path: "{0}"'.format(e.prefix_path))
5.101 + print(' data_path: "{0}"'.format(e.data_path))
5.102 +
5.103 +
5.104 +# test the module
5.105 +if __name__ == '__main__':
5.106 + test()
6.1 --- /dev/null Thu Jan 01 00:00:00 1970 +0000
6.2 +++ b/OpenSecurity/client/launch.py Mon Dec 02 14:02:05 2013 +0100
6.3 @@ -0,0 +1,287 @@
6.4 +#!/bin/env python
6.5 +# -*- coding: utf-8 -*-
6.6 +
6.7 +# ------------------------------------------------------------
6.8 +# opensecurity-launcher
6.9 +#
6.10 +# launches an application inside a VM
6.11 +#
6.12 +# Autor: Oliver Maurhart, <oliver.maurhart@ait.ac.at>
6.13 +#
6.14 +# Copyright (C) 2013 AIT Austrian Institute of Technology
6.15 +# AIT Austrian Institute of Technology GmbH
6.16 +# Donau-City-Strasse 1 | 1220 Vienna | Austria
6.17 +# http://www.ait.ac.at
6.18 +#
6.19 +# This program is free software; you can redistribute it and/or
6.20 +# modify it under the terms of the GNU General Public License
6.21 +# as published by the Free Software Foundation version 2.
6.22 +#
6.23 +# This program is distributed in the hope that it will be useful,
6.24 +# but WITHOUT ANY WARRANTY; without even the implied warranty of
6.25 +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
6.26 +# GNU General Public License for more details.
6.27 +#
6.28 +# You should have received a copy of the GNU General Public License
6.29 +# along with this program; if not, write to the Free Software
6.30 +# Foundation, Inc., 51 Franklin Street, Fifth Floor,
6.31 +# Boston, MA 02110-1301, USA.
6.32 +# ------------------------------------------------------------
6.33 +
6.34 +
6.35 +# ------------------------------------------------------------
6.36 +# imports
6.37 +
6.38 +import argparse
6.39 +import os
6.40 +import subprocess
6.41 +import sys
6.42 +
6.43 +from PyQt4 import QtCore
6.44 +from PyQt4 import QtGui
6.45 +
6.46 +# local
6.47 +from about import About
6.48 +from cygwin import Cygwin
6.49 +from environment import Environment
6.50 +import opensecurity_server
6.51 +
6.52 +
6.53 +# ------------------------------------------------------------
6.54 +# code
6.55 +
6.56 +
6.57 +class Chooser(QtGui.QDialog, object):
6.58 +
6.59 + """Ask the user what to launch."""
6.60 +
6.61 + def __init__(self, parent = None, flags = QtCore.Qt.WindowFlags(0)):
6.62 +
6.63 + super(Chooser, self).__init__(parent, flags)
6.64 + self.setWindowTitle('OpenSecuirty Launch Application')
6.65 + self.setup_ui()
6.66 +
6.67 + # known vms and applications
6.68 + self._apps, self_vms = [], []
6.69 +
6.70 + # positionate ourself central
6.71 + screen = QtGui.QDesktopWidget().screenGeometry()
6.72 + self.resize(self.geometry().width() * 1.25, self.geometry().height())
6.73 + size = self.geometry()
6.74 + self.move((screen.width() - size.width()) / 2, (screen.height() - size.height()) / 2)
6.75 +
6.76 + # refresh vm and command input
6.77 + self.refresh()
6.78 +
6.79 +
6.80 + def app_get(self):
6.81 + """The application of the user."""
6.82 + a = str(self._cbApplication.currentText())
6.83 + for app in self._apps:
6.84 + if a == app['name']:
6.85 + return app['command']
6.86 + return a
6.87 +
6.88 + app = property(app_get)
6.89 +
6.90 +
6.91 + def clicked_about(self):
6.92 + """clicked the about button"""
6.93 + dlgAbout = About()
6.94 + dlgAbout.exec_()
6.95 +
6.96 +
6.97 + def clicked_cancel(self):
6.98 + """clicked the cancel button"""
6.99 + self.reject()
6.100 +
6.101 +
6.102 + def clicked_ok(self):
6.103 + """clicked the ok button"""
6.104 + self.accept()
6.105 +
6.106 +
6.107 + def refresh(self):
6.108 + """load the known vms and commands and adjust input fields"""
6.109 +
6.110 + self._apps = opensecurity_server.query_apps()
6.111 + self._vms = opensecurity_server.query_vms()
6.112 +
6.113 + # add the VMs we know
6.114 + self._cbApplication.clear()
6.115 + for app in self._apps:
6.116 + self._cbApplication.addItem(app['name'])
6.117 +
6.118 + # add the commands we know
6.119 + self._cbVM.clear()
6.120 + for vm in self._vms:
6.121 + self._cbVM.addItem(vm['name'])
6.122 +
6.123 +
6.124 + def setup_ui(self):
6.125 + """Create the widgets."""
6.126 +
6.127 + lyMain = QtGui.QVBoxLayout(self)
6.128 + lyMain.setContentsMargins(8, 8, 8, 8)
6.129 +
6.130 + # content area: left pixmap, right text
6.131 + lyContent = QtGui.QHBoxLayout()
6.132 + lyMain.addLayout(lyContent)
6.133 +
6.134 + # pixmap
6.135 + lbPix = QtGui.QLabel()
6.136 + lbPix.setPixmap(QtGui.QPixmapCache.find('opensecurity_icon_64'))
6.137 + lyContent.addWidget(lbPix, 0, QtCore.Qt.Alignment(QtCore.Qt.AlignTop + QtCore.Qt.AlignHCenter))
6.138 + lyContent.addSpacing(16)
6.139 +
6.140 + # launch ...
6.141 + lyLaunch = QtGui.QGridLayout()
6.142 + lyContent.addLayout(lyLaunch)
6.143 + lbTitle = QtGui.QLabel('Specify details for application to launch.')
6.144 + lyLaunch.addWidget(lbTitle, 0, 0, 1, 2)
6.145 +
6.146 + lbVM = QtGui.QLabel('&VM-ID:')
6.147 + lyLaunch.addWidget(lbVM, 1, 0)
6.148 + self._cbVM = QtGui.QComboBox()
6.149 + self._cbVM.setEditable(True)
6.150 + self._cbVM.setInsertPolicy(QtGui.QComboBox.InsertAlphabetically)
6.151 + lyLaunch.addWidget(self._cbVM, 1, 1)
6.152 + lbVM.setBuddy(self._cbVM)
6.153 +
6.154 + lbApplication = QtGui.QLabel('&Application:')
6.155 + lyLaunch.addWidget(lbApplication, 2, 0)
6.156 + self._cbApplication = QtGui.QComboBox()
6.157 + self._cbApplication.setEditable(True)
6.158 + self._cbApplication.setInsertPolicy(QtGui.QComboBox.InsertAlphabetically)
6.159 + lyLaunch.addWidget(self._cbApplication, 2, 1)
6.160 + lbApplication.setBuddy(self._cbApplication)
6.161 +
6.162 + lyLaunch.addWidget(QtGui.QWidget(), 3, 0, 1, 2)
6.163 + lyLaunch.setColumnStretch(1, 1)
6.164 + lyLaunch.setRowStretch(3, 1)
6.165 +
6.166 + lyMain.addStretch(1)
6.167 +
6.168 + # buttons
6.169 + lyButton = QtGui.QHBoxLayout()
6.170 + lyMain.addLayout(lyButton)
6.171 +
6.172 + lyButton.addStretch(1)
6.173 + btnOk = QtGui.QPushButton('&Ok', self)
6.174 + btnOk.setDefault(True)
6.175 + btnOk.setMinimumWidth(100)
6.176 + lyButton.addWidget(btnOk)
6.177 + btnCancel = QtGui.QPushButton('&Cancel', self)
6.178 + btnCancel.setMinimumWidth(100)
6.179 + lyButton.addWidget(btnCancel)
6.180 + btnAbout = QtGui.QPushButton('&About', self)
6.181 + btnAbout.setMinimumWidth(100)
6.182 + lyButton.addWidget(btnAbout)
6.183 +
6.184 + button_width = max(btnOk.width(), btnCancel.width(), btnAbout.width())
6.185 + btnOk.setMinimumWidth(button_width)
6.186 + btnCancel.setMinimumWidth(button_width)
6.187 + btnAbout.setMinimumWidth(button_width)
6.188 +
6.189 + # reduce to the max
6.190 + self.resize(lyMain.minimumSize())
6.191 +
6.192 + # connectors
6.193 + btnOk.clicked.connect(self.clicked_ok)
6.194 + btnCancel.clicked.connect(self.clicked_cancel)
6.195 + btnAbout.clicked.connect(self.clicked_about)
6.196 +
6.197 +
6.198 + def user_get(self):
6.199 + """The user of the vm of choice."""
6.200 + v = str(self._cbVM.currentText())
6.201 + for vm in self._vms:
6.202 + if v == vm['name']:
6.203 + return vm['user']
6.204 + return v
6.205 +
6.206 + user = property(user_get)
6.207 +
6.208 +
6.209 + def vm_get(self):
6.210 + """The vm of choice."""
6.211 + v = str(self._cbVM.currentText())
6.212 + for vm in self._vms:
6.213 + if v == vm['name']:
6.214 + return vm['ip']
6.215 + return v
6.216 +
6.217 + vm = property(vm_get)
6.218 +
6.219 +
6.220 +def ask_user():
6.221 + """ask the user for VM and app to start"""
6.222 +
6.223 + # launch Qt
6.224 + app = QtGui.QApplication(sys.argv)
6.225 +
6.226 + # prebuild the pixmap cache: fetch all jpg, png and jpeg images and load them
6.227 + image_path = os.path.join(Environment("OpenSecurity").data_path, '..', 'gfx')
6.228 + for file in os.listdir(image_path):
6.229 + if file.lower().rpartition('.')[2] in ('jpg', 'png', 'jpeg'):
6.230 + QtGui.QPixmapCache.insert(file.lower().rpartition('.')[0], QtGui.QPixmap(os.path.join(image_path, file)))
6.231 +
6.232 + # we should have now our application icon
6.233 + app.setWindowIcon(QtGui.QIcon(QtGui.QPixmapCache.find('opensecurity_icon_64')))
6.234 +
6.235 + # pop up the dialog
6.236 + dlg = Chooser()
6.237 + dlg.show()
6.238 + app.exec_()
6.239 +
6.240 + if dlg.result() == QtGui.QDialog.Accepted:
6.241 + return dlg.user, dlg.vm, dlg.app
6.242 +
6.243 + return '', '', ''
6.244 +
6.245 +
6.246 +def main():
6.247 + """entry point"""
6.248 +
6.249 + # parse command line
6.250 + parser = argparse.ArgumentParser(description = 'OpenSecurity Launcher: run application in VM')
6.251 + parser.add_argument('user', metavar='USER', help='USER on Virtual Machine', nargs='?', type=str, default='')
6.252 + parser.add_argument('ip', metavar='IP', help='IP of Virtual Machine', nargs='?', type=str, default='')
6.253 + parser.add_argument('command', metavar='COMMAND', help='Full path of command and arguments to start inside VM', nargs='?', type=str, default='')
6.254 + args = parser.parse_args()
6.255 +
6.256 + # we must have at least all or none set
6.257 + set_user = args.user != ''
6.258 + set_ip = args.ip != ''
6.259 + set_command = args.command != ''
6.260 + set_ALL = set_user and set_ip and set_command
6.261 + set_NONE = (not set_user) and (not set_ip) and (not set_command)
6.262 + if (not set_ALL) and (not set_NONE):
6.263 + sys.stderr.write("Please specify user, ip and command or none.\n")
6.264 + sys.stderr.write("Type '--help' for help.\n")
6.265 + sys.exit(1)
6.266 +
6.267 + # check if we need to ask the user
6.268 + if set_NONE:
6.269 + args.user, args.ip, args.command = ask_user()
6.270 +
6.271 + # still no IP? --> no chance, over and out!
6.272 + if args.ip == '':
6.273 + sys.exit(0)
6.274 +
6.275 + # ensure we have our X11 running
6.276 + Cygwin.start_X11()
6.277 +
6.278 + # the SSH command
6.279 + user_at_guest = args.user + '@' + args.ip
6.280 + ssh = 'DISPLAY=:0 /usr/bin/ssh -Y ' + user_at_guest + ' ' + args.command
6.281 + print(ssh)
6.282 +
6.283 + # off we go!
6.284 + Cygwin()(['/bin/bash', '--login', '-i', '-c', ssh], None, None, None)
6.285 +
6.286 +
6.287 +# start
6.288 +if __name__ == "__main__":
6.289 + main()
6.290 +
7.1 --- /dev/null Thu Jan 01 00:00:00 1970 +0000
7.2 +++ b/OpenSecurity/client/opensecurity_client_restful_server.py Mon Dec 02 14:02:05 2013 +0100
7.3 @@ -0,0 +1,215 @@
7.4 +#!/bin/env python
7.5 +# -*- coding: utf-8 -*-
7.6 +
7.7 +# ------------------------------------------------------------
7.8 +# opensecurity_client_restful_server
7.9 +#
7.10 +# the OpenSecurity client RESTful server
7.11 +#
7.12 +# Autor: Oliver Maurhart, <oliver.maurhart@ait.ac.at>
7.13 +#
7.14 +# Copyright (C) 2013 AIT Austrian Institute of Technology
7.15 +# AIT Austrian Institute of Technology GmbH
7.16 +# Donau-City-Strasse 1 | 1220 Vienna | Austria
7.17 +# http://www.ait.ac.at
7.18 +#
7.19 +# This program is free software; you can redistribute it and/or
7.20 +# modify it under the terms of the GNU General Public License
7.21 +# as published by the Free Software Foundation version 2.
7.22 +#
7.23 +# This program is distributed in the hope that it will be useful,
7.24 +# but WITHOUT ANY WARRANTY; without even the implied warranty of
7.25 +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
7.26 +# GNU General Public License for more details.
7.27 +#
7.28 +# You should have received a copy of the GNU General Public License
7.29 +# along with this program; if not, write to the Free Software
7.30 +# Foundation, Inc., 51 Franklin Street, Fifth Floor,
7.31 +# Boston, MA 02110-1301, USA.
7.32 +# ------------------------------------------------------------
7.33 +
7.34 +
7.35 +# ------------------------------------------------------------
7.36 +# imports
7.37 +
7.38 +import os
7.39 +import os.path
7.40 +import subprocess
7.41 +import sys
7.42 +import web
7.43 +
7.44 +# local
7.45 +from environment import Environment
7.46 +import opensecurity_server
7.47 +
7.48 +
7.49 +# ------------------------------------------------------------
7.50 +# const
7.51 +
7.52 +
7.53 +__version__ = "0.1"
7.54 +
7.55 +
7.56 +"""All the URLs we know mapping to class handler"""
7.57 +opensecurity_urls = (
7.58 + '/application', 'os_application',
7.59 + '/credentials', 'os_credentials',
7.60 + '/password', 'os_password',
7.61 + '/', 'os_root'
7.62 +)
7.63 +
7.64 +
7.65 +# ------------------------------------------------------------
7.66 +# code
7.67 +
7.68 +
7.69 +class os_application:
7.70 + """OpenSecurity '/application' handler.
7.71 +
7.72 + This is called on GET /application?vm=VM-ID&app=APP-ID
7.73 + This tries to access the vm identified with the label VM-ID
7.74 + and launched the application identified APP-ID
7.75 + """
7.76 +
7.77 + def GET(self):
7.78 +
7.79 + # pick the arguments
7.80 + args = web.input()
7.81 +
7.82 + # we _need_ a vm
7.83 + if not "vm" in args:
7.84 + raise web.badrequest()
7.85 +
7.86 + # we _need_ a app
7.87 + if not "app" in args:
7.88 + raise web.badrequest()
7.89 +
7.90 + apps = opensecurity_server.query_apps()
7.91 + vms = opensecurity_server.query_vms()
7.92 +
7.93 + # check if we do have valid vm
7.94 + v = [v for v in vms if v['name'] == args.vm]
7.95 + if len(v) == 0:
7.96 + raise web.notfound('vm not found')
7.97 + v = v[0]
7.98 +
7.99 + # check if we do have a valid app
7.100 + a = [a for a in apps if a['name'] == args.app]
7.101 + if len(a) == 0:
7.102 + raise web.notfound('app not found')
7.103 + a = a[0]
7.104 +
7.105 + # invoke launch with
7.106 + res = "starting: launch " + v['user'] + " " + v['ip'] + " " + a['command']
7.107 +
7.108 + launch_image = os.path.join(sys.path[0], 'launch.py')
7.109 + process_command = [sys.executable, launch_image, v['user'], v['ip'], a['command']]
7.110 + process = subprocess.Popen(process_command, shell = False, stdout = subprocess.PIPE)
7.111 + result = process.communicate()[0]
7.112 + if process.returncode != 0:
7.113 + return 'Launch of application aborted.'
7.114 +
7.115 + return result
7.116 +
7.117 +
7.118 +class os_credentials:
7.119 + """OpenSecurity '/credentials' handler.
7.120 +
7.121 + This is called on GET /credentials?text=TEXT.
7.122 + Ideally this should pop up a user dialog to insert his
7.123 + credentials based the given TEXT.
7.124 + """
7.125 +
7.126 + def GET(self):
7.127 +
7.128 + # pick the arguments
7.129 + args = web.input()
7.130 +
7.131 + # we _need_ a device id
7.132 + if not "text" in args:
7.133 + raise web.badrequest()
7.134 +
7.135 + # invoke the user dialog as a subprocess
7.136 + dlg_credentials_image = os.path.join(sys.path[0], 'opensecurity_dialog.py')
7.137 + process_command = [sys.executable, dlg_credentials_image, 'credentials', args.text]
7.138 + process = subprocess.Popen(process_command, shell = False, stdout = subprocess.PIPE)
7.139 + result = process.communicate()[0]
7.140 + if process.returncode != 0:
7.141 + return 'Credentials request has been aborted.'
7.142 +
7.143 + return result
7.144 +
7.145 +
7.146 +class os_password:
7.147 + """OpenSecurity '/password' handler.
7.148 +
7.149 + This is called on GET /password?text=TEXT.
7.150 + Ideally this should pop up a user dialog to insert his
7.151 + password based device name.
7.152 + """
7.153 +
7.154 + def GET(self):
7.155 +
7.156 + # pick the arguments
7.157 + args = web.input()
7.158 +
7.159 + # we _need_ a device id
7.160 + if not "text" in args:
7.161 + raise web.badrequest()
7.162 +
7.163 + # invoke the user dialog as a subprocess
7.164 + dlg_credentials_image = os.path.join(sys.path[0], 'opensecurity_dialog.py')
7.165 + process_command = [sys.executable, dlg_credentials_image, 'password', args.text]
7.166 + process = subprocess.Popen(process_command, shell = False, stdout = subprocess.PIPE)
7.167 + result = process.communicate()[0]
7.168 + if process.returncode != 0:
7.169 + return 'password request has been aborted.'
7.170 +
7.171 + return result
7.172 +
7.173 +
7.174 +class os_root:
7.175 + """OpenSecurity '/' handler"""
7.176 +
7.177 + def GET(self):
7.178 +
7.179 + res = "OpenSecurity-Client RESTFul Server { \"version\": \"%s\" }" % __version__
7.180 +
7.181 + # add some sample links
7.182 + res = res + """
7.183 +
7.184 +USAGE EXAMPLES:
7.185 +
7.186 +Request a password:
7.187 + (copy paste this into your browser's address field after the host:port)
7.188 +
7.189 + /password?text=Give+me+a+password+for+device+%22My+USB+Drive%22+(ID%3A+32090-AAA-X0)
7.190 +
7.191 + (eg.: http://127.0.0.1:8090/password?text=Give+me+a+password+for+device+%22My+USB+Drive%22+(ID%3A+32090-AAA-X0))
7.192 + NOTE: check yout taskbar, the dialog window may not pop up in front of your browser window.
7.193 +
7.194 +
7.195 +Request a combination of user and password:
7.196 + (copy paste this into your browser's address field after the host:port)
7.197 +
7.198 + /credentials?text=Tell+the+NSA+which+credentials+to+use+in+order+to+avoid+hacking+noise+on+wire.
7.199 +
7.200 + (eg.: http://127.0.0.1:8090/credentials?text=Tell+the+NSA+which+credentials+to+use+in+order+to+avoid+hacking+noise+on+wire.)
7.201 + NOTE: check yout taskbar, the dialog window may not pop up in front of your browser window.
7.202 +
7.203 +
7.204 +Start a Browser:
7.205 + (copy paste this into your browser's address field after the host:port)
7.206 +
7.207 + /application?vm=Debian+7&app=Browser
7.208 +
7.209 + (e.g. http://127.0.0.1:8090/application?vm=Debian+7&app=Browser)
7.210 + """
7.211 +
7.212 + return res
7.213 +
7.214 +
7.215 +# start
7.216 +if __name__ == "__main__":
7.217 + server = web.application(opensecurity_urls, globals())
7.218 + server.run()
8.1 --- /dev/null Thu Jan 01 00:00:00 1970 +0000
8.2 +++ b/OpenSecurity/client/opensecurity_dialog.py Mon Dec 02 14:02:05 2013 +0100
8.3 @@ -0,0 +1,93 @@
8.4 +#!/bin/env python
8.5 +# -*- coding: utf-8 -*-
8.6 +
8.7 +# ------------------------------------------------------------
8.8 +# opensecurity-dialog
8.9 +#
8.10 +# an opensecurity dialog
8.11 +#
8.12 +# Autor: Oliver Maurhart, <oliver.maurhart@ait.ac.at>
8.13 +#
8.14 +# Copyright (C) 2013 AIT Austrian Institute of Technology
8.15 +# AIT Austrian Institute of Technology GmbH
8.16 +# Donau-City-Strasse 1 | 1220 Vienna | Austria
8.17 +# http://www.ait.ac.at
8.18 +#
8.19 +# This program is free software; you can redistribute it and/or
8.20 +# modify it under the terms of the GNU General Public License
8.21 +# as published by the Free Software Foundation version 2.
8.22 +#
8.23 +# This program is distributed in the hope that it will be useful,
8.24 +# but WITHOUT ANY WARRANTY; without even the implied warranty of
8.25 +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
8.26 +# GNU General Public License for more details.
8.27 +#
8.28 +# You should have received a copy of the GNU General Public License
8.29 +# along with this program; if not, write to the Free Software
8.30 +# Foundation, Inc., 51 Franklin Street, Fifth Floor,
8.31 +# Boston, MA 02110-1301, USA.
8.32 +# ------------------------------------------------------------
8.33 +
8.34 +
8.35 +# ------------------------------------------------------------
8.36 +# imports
8.37 +
8.38 +import argparse
8.39 +import os
8.40 +import sys
8.41 +
8.42 +from PyQt4 import QtCore
8.43 +from PyQt4 import QtGui
8.44 +
8.45 +# local
8.46 +from credentials import Credentials
8.47 +from environment import Environment
8.48 +from password import Password
8.49 +
8.50 +
8.51 +# ------------------------------------------------------------
8.52 +# code
8.53 +
8.54 +
8.55 +def main():
8.56 +
8.57 + # parse command line
8.58 + parser = argparse.ArgumentParser(description = 'OpenSecurity Dialog.')
8.59 + parser.add_argument('mode', metavar='MODE', help='dialog mode: \'password\' or \'credentials\'')
8.60 + parser.add_argument('text', metavar='TEXT', help='text to show')
8.61 + args = parser.parse_args()
8.62 +
8.63 + app = QtGui.QApplication(sys.argv)
8.64 +
8.65 + # prebuild the pixmap cache: fetch all jpg, png and jpeg images and load them
8.66 + data_path = Environment("OpenSecurity").data_path
8.67 + image_path = os.path.join(data_path, '..', 'gfx')
8.68 + for file in os.listdir(image_path):
8.69 + if file.lower().rpartition('.')[2] in ('jpg', 'png', 'jpeg'):
8.70 + QtGui.QPixmapCache.insert(file.lower().rpartition('.')[0], QtGui.QPixmap(os.path.join(image_path, file)))
8.71 +
8.72 + # we should have now our application icon
8.73 + app.setWindowIcon(QtGui.QIcon(QtGui.QPixmapCache.find('opensecurity_icon_64')))
8.74 +
8.75 + if args.mode == 'password':
8.76 + dlg = Password(args.text)
8.77 +
8.78 + if args.mode == 'credentials':
8.79 + dlg = Credentials(args.text)
8.80 +
8.81 + # pop up the dialog
8.82 + dlg.show()
8.83 + app.exec_()
8.84 +
8.85 + # give proper result code
8.86 + if dlg.result() == QtGui.QDialog.Accepted:
8.87 + res = 0
8.88 + else:
8.89 + res = 1
8.90 + sys.exit(res)
8.91 +
8.92 +
8.93 +# start
8.94 +if __name__ == "__main__":
8.95 + main()
8.96 +
9.1 --- /dev/null Thu Jan 01 00:00:00 1970 +0000
9.2 +++ b/OpenSecurity/client/opensecurity_server.py Mon Dec 02 14:02:05 2013 +0100
9.3 @@ -0,0 +1,71 @@
9.4 +#!/bin/env python
9.5 +# -*- coding: utf-8 -*-
9.6 +
9.7 +# ------------------------------------------------------------
9.8 +# opensecurity-server
9.9 +#
9.10 +# talk to the opensecurity server
9.11 +#
9.12 +# Autor: Oliver Maurhart, <oliver.maurhart@ait.ac.at>
9.13 +#
9.14 +# Copyright (C) 2013 AIT Austrian Institute of Technology
9.15 +# AIT Austrian Institute of Technology GmbH
9.16 +# Donau-City-Strasse 1 | 1220 Vienna | Austria
9.17 +# http://www.ait.ac.at
9.18 +#
9.19 +# This program is free software; you can redistribute it and/or
9.20 +# modify it under the terms of the GNU General Public License
9.21 +# as published by the Free Software Foundation version 2.
9.22 +#
9.23 +# This program is distributed in the hope that it will be useful,
9.24 +# but WITHOUT ANY WARRANTY; without even the implied warranty of
9.25 +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
9.26 +# GNU General Public License for more details.
9.27 +#
9.28 +# You should have received a copy of the GNU General Public License
9.29 +# along with this program; if not, write to the Free Software
9.30 +# Foundation, Inc., 51 Franklin Street, Fifth Floor,
9.31 +# Boston, MA 02110-1301, USA.
9.32 +# ------------------------------------------------------------
9.33 +
9.34 +# ------------------------------------------------------------
9.35 +# import
9.36 +
9.37 +from pprint import PrettyPrinter
9.38 +
9.39 +
9.40 +# ------------------------------------------------------------
9.41 +# code
9.42 +
9.43 +def query_apps():
9.44 + """get the list of known apps"""
9.45 +
9.46 + # TODO: REPLACE THIS HARDCODED STUFF WITH REAL CODE TO THE OS SERVER
9.47 + apps = [
9.48 + { 'vm': 'Debian 7', 'name': 'Browser', 'command': '/usr/bin/iceweasel'},
9.49 + { 'vm': 'Debian 7', 'name': 'VLC', 'command': '/usr/bin/vlc'}
9.50 + ]
9.51 +
9.52 + return apps
9.53 +
9.54 +
9.55 +def query_vms():
9.56 + """get the list of registered vms, their ip and the prefered user"""
9.57 +
9.58 + # TODO: REPLACE THIS HARDCODED STUFF WITH REAL CODE TO THE OS SERVER
9.59 + vms = [
9.60 + { 'user': 'user', 'name': 'Debian 7', 'ip': '192.168.56.101'},
9.61 + { 'user': 'user', 'name': 'Anit-Virus VM', 'ip': '192.168.56.101'}
9.62 + ]
9.63 +
9.64 + return vms
9.65 +
9.66 +
9.67 +# start
9.68 +if __name__ == "__main__":
9.69 + print("known apps: ")
9.70 + PrettyPrinter().pprint(query_apps())
9.71 + print("known vms: ")
9.72 + PrettyPrinter().pprint(query_vms())
9.73 +
9.74 +
10.1 --- /dev/null Thu Jan 01 00:00:00 1970 +0000
10.2 +++ b/OpenSecurity/client/opensecurity_tray.py Mon Dec 02 14:02:05 2013 +0100
10.3 @@ -0,0 +1,131 @@
10.4 +#!/bin/env python
10.5 +# -*- coding: utf-8 -*-
10.6 +
10.7 +# ------------------------------------------------------------
10.8 +# opensecurity-dialog
10.9 +#
10.10 +# an opensecurity dialog
10.11 +#
10.12 +# Autor: Oliver Maurhart, <oliver.maurhart@ait.ac.at>
10.13 +#
10.14 +# Copyright (C) 2013 AIT Austrian Institute of Technology
10.15 +# AIT Austrian Institute of Technology GmbH
10.16 +# Donau-City-Strasse 1 | 1220 Vienna | Austria
10.17 +# http://www.ait.ac.at
10.18 +#
10.19 +# This program is free software; you can redistribute it and/or
10.20 +# modify it under the terms of the GNU General Public License
10.21 +# as published by the Free Software Foundation version 2.
10.22 +#
10.23 +# This program is distributed in the hope that it will be useful,
10.24 +# but WITHOUT ANY WARRANTY; without even the implied warranty of
10.25 +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
10.26 +# GNU General Public License for more details.
10.27 +#
10.28 +# You should have received a copy of the GNU General Public License
10.29 +# along with this program; if not, write to the Free Software
10.30 +# Foundation, Inc., 51 Franklin Street, Fifth Floor,
10.31 +# Boston, MA 02110-1301, USA.
10.32 +# ------------------------------------------------------------
10.33 +
10.34 +
10.35 +# ------------------------------------------------------------
10.36 +# imports
10.37 +
10.38 +import argparse
10.39 +import os
10.40 +import subprocess
10.41 +import sys
10.42 +
10.43 +from PyQt4 import QtCore
10.44 +from PyQt4 import QtGui
10.45 +
10.46 +# local
10.47 +from about import About
10.48 +from environment import Environment
10.49 +
10.50 +
10.51 +# ------------------------------------------------------------
10.52 +# code
10.53 +
10.54 +
10.55 +class OpenSecurityTrayIcon(QtGui.QSystemTrayIcon):
10.56 +
10.57 + """This is the OpenSecuirty Tray Icon"""
10.58 +
10.59 + def __init__(self, icon, parent=None):
10.60 +
10.61 + super(OpenSecurityTrayIcon, self).__init__(icon, parent)
10.62 + self.setup_ui()
10.63 +
10.64 +
10.65 + def clicked_about(self):
10.66 + """clicked about"""
10.67 + dlgAbout = About()
10.68 + dlgAbout.exec_()
10.69 +
10.70 +
10.71 + def clicked_exit(self):
10.72 + """clicked exit"""
10.73 + sys.exit(0)
10.74 +
10.75 +
10.76 + def clicked_launch_application(self):
10.77 + """clicked the launch an application"""
10.78 + dlg_launch_image = os.path.join(sys.path[0], 'launch.pyw')
10.79 + process_command = [sys.executable, dlg_launch_image]
10.80 + print(process_command)
10.81 + process = subprocess.Popen(process_command, shell = False)
10.82 + process.communicate()
10.83 +
10.84 +
10.85 + def clicked_refresh(self):
10.86 + """clicked refresh"""
10.87 + self.setup_ui()
10.88 +
10.89 +
10.90 + def setup_ui(self):
10.91 + """create the user interface
10.92 + As for the system tray this is 'just' the context menu.
10.93 + """
10.94 +
10.95 + # define the tray icon menu
10.96 + menu = QtGui.QMenu(self.parent())
10.97 + self.setContextMenu(menu)
10.98 +
10.99 + # add known apps
10.100 +
10.101 + # add standard menu items
10.102 + cAcLaunch = menu.addAction(QtGui.QIcon(QtGui.QPixmapCache.find('opensecurity_icon_64')), 'Lauch Application')
10.103 + menu.addSeparator()
10.104 + cAcRefresh = menu.addAction('Refresh')
10.105 + cAcAbout = menu.addAction("About")
10.106 + cAcExit = menu.addAction("Exit")
10.107 +
10.108 + cAcLaunch.triggered.connect(self.clicked_launch_application)
10.109 + cAcRefresh.triggered.connect(self.clicked_refresh)
10.110 + cAcAbout.triggered.connect(self.clicked_about)
10.111 + cAcExit.triggered.connect(self.clicked_exit)
10.112 +
10.113 +
10.114 +def main():
10.115 +
10.116 + app = QtGui.QApplication(sys.argv)
10.117 +
10.118 + # prebuild the pixmap cache: fetch all jpg, png and jpeg images and load them
10.119 + image_path = os.path.join(Environment("OpenSecurity").data_path, '..', 'gfx')
10.120 + for file in os.listdir(image_path):
10.121 + if file.lower().rpartition('.')[2] in ('jpg', 'png', 'jpeg'):
10.122 + QtGui.QPixmapCache.insert(file.lower().rpartition('.')[0], QtGui.QPixmap(os.path.join(image_path, file)))
10.123 +
10.124 + w = QtGui.QWidget()
10.125 + trayIcon = OpenSecurityTrayIcon(QtGui.QIcon(QtGui.QPixmapCache.find('opensecurity_icon_64')), w)
10.126 +
10.127 + trayIcon.show()
10.128 + sys.exit(app.exec_())
10.129 +
10.130 +
10.131 +# start
10.132 +if __name__ == "__main__":
10.133 + main()
10.134 +
11.1 --- /dev/null Thu Jan 01 00:00:00 1970 +0000
11.2 +++ b/OpenSecurity/client/password.py Mon Dec 02 14:02:05 2013 +0100
11.3 @@ -0,0 +1,150 @@
11.4 +#!/bin/env python
11.5 +# -*- coding: utf-8 -*-
11.6 +
11.7 +# ------------------------------------------------------------
11.8 +# password-dialog
11.9 +#
11.10 +# ask the user a password
11.11 +#
11.12 +# Autor: Oliver Maurhart, <oliver.maurhart@ait.ac.at>
11.13 +#
11.14 +# Copyright (C) 2013 AIT Austrian Institute of Technology
11.15 +# AIT Austrian Institute of Technology GmbH
11.16 +# Donau-City-Strasse 1 | 1220 Vienna | Austria
11.17 +# http://www.ait.ac.at
11.18 +#
11.19 +# This program is free software; you can redistribute it and/or
11.20 +# modify it under the terms of the GNU General Public License
11.21 +# as published by the Free Software Foundation version 2.
11.22 +#
11.23 +# This program is distributed in the hope that it will be useful,
11.24 +# but WITHOUT ANY WARRANTY; without even the implied warranty of
11.25 +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
11.26 +# GNU General Public License for more details.
11.27 +#
11.28 +# You should have received a copy of the GNU General Public License
11.29 +# along with this program; if not, write to the Free Software
11.30 +# Foundation, Inc., 51 Franklin Street, Fifth Floor,
11.31 +# Boston, MA 02110-1301, USA.
11.32 +# ------------------------------------------------------------
11.33 +
11.34 +
11.35 +# ------------------------------------------------------------
11.36 +# imports
11.37 +
11.38 +import sys
11.39 +
11.40 +from PyQt4 import QtCore
11.41 +from PyQt4 import QtGui
11.42 +
11.43 +# local
11.44 +from about import About
11.45 +
11.46 +# ------------------------------------------------------------
11.47 +# code
11.48 +
11.49 +
11.50 +class Password(QtGui.QDialog):
11.51 +
11.52 + """Ask the user for a password."""
11.53 +
11.54 + def __init__(self, text, parent = None, flags = QtCore.Qt.WindowFlags(0)):
11.55 +
11.56 + # super call and widget init
11.57 + super(Password, self).__init__(parent, flags)
11.58 + self.setWindowTitle('OpenSecuirty Password Request')
11.59 + self.setup_ui()
11.60 +
11.61 + # positionate ourself central
11.62 + screen = QtGui.QDesktopWidget().screenGeometry()
11.63 + self.resize(self.geometry().width() * 1.25, self.geometry().height())
11.64 + size = self.geometry()
11.65 + self.move((screen.width() - size.width()) / 2, (screen.height() - size.height()) / 2)
11.66 +
11.67 + # fix up text
11.68 + self.lbText.setText(text)
11.69 +
11.70 +
11.71 + def clicked_about(self):
11.72 + """clicked the about button"""
11.73 + dlgAbout = About()
11.74 + dlgAbout.exec_()
11.75 +
11.76 +
11.77 + def clicked_cancel(self):
11.78 + """clicked the cancel button"""
11.79 + self.reject()
11.80 +
11.81 +
11.82 + def clicked_ok(self):
11.83 + """clicked the ok button"""
11.84 + sys.stdout.write('{ ')
11.85 + sys.stdout.write('\'password\': \'')
11.86 + sys.stdout.write(self.edPassword.text())
11.87 + sys.stdout.write('\' ')
11.88 + sys.stdout.write('}\n')
11.89 + self.accept()
11.90 +
11.91 +
11.92 + def setup_ui(self):
11.93 +
11.94 + """Create the widgets."""
11.95 +
11.96 + lyMain = QtGui.QVBoxLayout(self)
11.97 + lyMain.setContentsMargins(8, 8, 8, 8)
11.98 +
11.99 + # content area: left pixmap, right text
11.100 + lyContent = QtGui.QHBoxLayout()
11.101 + lyMain.addLayout(lyContent)
11.102 +
11.103 + # pixmap
11.104 + lbPix = QtGui.QLabel()
11.105 + lbPix.setPixmap(QtGui.QPixmapCache.find('opensecurity_icon_64'))
11.106 + lyContent.addWidget(lbPix, 0, QtCore.Qt.Alignment(QtCore.Qt.AlignTop + QtCore.Qt.AlignHCenter))
11.107 + lyContent.addSpacing(16)
11.108 +
11.109 + # text ...
11.110 + lyText = QtGui.QVBoxLayout()
11.111 + lyContent.addLayout(lyText)
11.112 + self.lbText = QtGui.QLabel()
11.113 + lyText.addWidget(self.lbText)
11.114 + lyPassword = QtGui.QHBoxLayout()
11.115 + lyText.addLayout(lyPassword)
11.116 + lbPassword = QtGui.QLabel('&Password:')
11.117 + lyPassword.addWidget(lbPassword)
11.118 + self.edPassword = QtGui.QLineEdit()
11.119 + self.edPassword.setEchoMode(QtGui.QLineEdit.Password)
11.120 + lyPassword.addWidget(self.edPassword)
11.121 + lbPassword.setBuddy(self.edPassword)
11.122 + lyText.addStretch(1)
11.123 +
11.124 + lyMain.addStretch(1)
11.125 +
11.126 + # buttons
11.127 + lyButton = QtGui.QHBoxLayout()
11.128 + lyMain.addLayout(lyButton)
11.129 +
11.130 + lyButton.addStretch(1)
11.131 + btnOk = QtGui.QPushButton('&Ok', self)
11.132 + btnOk.setDefault(True)
11.133 + btnOk.setMinimumWidth(100)
11.134 + lyButton.addWidget(btnOk)
11.135 + btnCancel = QtGui.QPushButton('&Cancel', self)
11.136 + btnCancel.setMinimumWidth(100)
11.137 + lyButton.addWidget(btnCancel)
11.138 + btnAbout = QtGui.QPushButton('&About', self)
11.139 + btnAbout.setMinimumWidth(100)
11.140 + lyButton.addWidget(btnAbout)
11.141 +
11.142 + button_width = max(btnOk.width(), btnCancel.width(), btnAbout.width())
11.143 + btnOk.setMinimumWidth(button_width)
11.144 + btnCancel.setMinimumWidth(button_width)
11.145 + btnAbout.setMinimumWidth(button_width)
11.146 +
11.147 + # reduce to the max
11.148 + self.resize(lyMain.minimumSize())
11.149 +
11.150 + # connectors
11.151 + btnOk.clicked.connect(self.clicked_ok)
11.152 + btnCancel.clicked.connect(self.clicked_cancel)
11.153 + btnAbout.clicked.connect(self.clicked_about)
12.1 --- /dev/null Thu Jan 01 00:00:00 1970 +0000
12.2 +++ b/OpenSecurity/cygwin/Cygwin.vbs Mon Dec 02 14:02:05 2013 +0100
12.3 @@ -0,0 +1,50 @@
12.4 +Option Explicit
12.5 +
12.6 +' ------------------------------------------------------------
12.7 +' start a cygwin shell
12.8 +'
12.9 +' Autor: Oliver Maurhart, <oliver.maurhart@ait.ac.at>
12.10 +'
12.11 +' Copyright (C) 2013 AIT Austrian Institute of Technology
12.12 +' AIT Austrian Institute of Technology GmbH
12.13 +' Donau-City-Strasse 1 | 1220 Vienna | Austria
12.14 +' http://www.ait.ac.at
12.15 +' ------------------------------------------------------------
12.16 +
12.17 +' ------------------------------------------------------------
12.18 +' This starts a cygwin shell within a relocatable cygwin
12.19 +' folder but without a DOS shell window prior. The idea is
12.20 +' to do it like in this BAT snippet below:
12.21 +'
12.22 +' @echo off
12.23 +' SET INSTALL_DRIVE=%~d0
12.24 +' SET INSTALL_FOLDER=%~p0
12.25 +' SET CYGWIN_BIN="%INSTALL_DRIVE%%INSTALL_FOLDER%bin"
12.26 +' SET CYGWIN_BASH="%INSTALL_DRIVE%%INSTALL_FOLDER%bin\bash"
12.27 +' %INSTALL_DRIVE%
12.28 +' chdir "%CYGWIN_BIN%"
12.29 +' start mintty.exe %CYGWIN_BASH% --login -i
12.30 +'
12.31 +' ------------------------------------------------------------
12.32 +
12.33 +' setup the basic objects
12.34 +Dim cFSO
12.35 +Dim cShell
12.36 +Set cShell = CreateObject("WScript.Shell")
12.37 +Set cFSO = CreateObject("Scripting.FileSystemObject")
12.38 +
12.39 +' parse script location and cd into folder
12.40 +Dim sPath
12.41 +sPath = Wscript.ScriptFullName
12.42 +sPath = cFSO.GetAbsolutePathName(sPath)
12.43 +sPath = cFSO.GetParentFolderName(sPath)
12.44 +cShell.CurrentDirectory = sPath
12.45 +
12.46 +' locations of mintty and bash
12.47 +Dim sMinTTYPath
12.48 +Dim sBashPath
12.49 +sMinTTYPath = """" & sPath & "\bin\mintty" & """"
12.50 +sBashPath = """" & sPath & "\bin\bash" & """"
12.51 +
12.52 +' start a cygwin shell
12.53 +cShell.Run sMinTTYPath & " " & sBashPath & " --login -i", 1, false
13.1 --- /dev/null Thu Jan 01 00:00:00 1970 +0000
13.2 +++ b/OpenSecurity/cygwin/content.txt Mon Dec 02 14:02:05 2013 +0100
13.3 @@ -0,0 +1,10 @@
13.4 +Content of Opensecurity/cygwin
13.5 +==============================
13.6 +
13.7 +This folder contains a complete Cygwin - 32 Bit with at least these packages installed:
13.8 +
13.9 + - bash
13.10 + - coreutils
13.11 + - openssh
13.12 + - xinit
13.13 + - genisofs
14.1 Binary file OpenSecurity/cygwin/setup-x86.exe has changed
15.1 --- /dev/null Thu Jan 01 00:00:00 1970 +0000
15.2 +++ b/OpenSecurity/cygwin64/Cygwin.vbs Mon Dec 02 14:02:05 2013 +0100
15.3 @@ -0,0 +1,50 @@
15.4 +Option Explicit
15.5 +
15.6 +' ------------------------------------------------------------
15.7 +' start a cygwin shell
15.8 +'
15.9 +' Autor: Oliver Maurhart, <oliver.maurhart@ait.ac.at>
15.10 +'
15.11 +' Copyright (C) 2013 AIT Austrian Institute of Technology
15.12 +' AIT Austrian Institute of Technology GmbH
15.13 +' Donau-City-Strasse 1 | 1220 Vienna | Austria
15.14 +' http://www.ait.ac.at
15.15 +' ------------------------------------------------------------
15.16 +
15.17 +' ------------------------------------------------------------
15.18 +' This starts a cygwin shell within a relocatable cygwin
15.19 +' folder but without a DOS shell window prior. The idea is
15.20 +' to do it like in this BAT snippet below:
15.21 +'
15.22 +' @echo off
15.23 +' SET INSTALL_DRIVE=%~d0
15.24 +' SET INSTALL_FOLDER=%~p0
15.25 +' SET CYGWIN_BIN="%INSTALL_DRIVE%%INSTALL_FOLDER%bin"
15.26 +' SET CYGWIN_BASH="%INSTALL_DRIVE%%INSTALL_FOLDER%bin\bash"
15.27 +' %INSTALL_DRIVE%
15.28 +' chdir "%CYGWIN_BIN%"
15.29 +' start mintty.exe %CYGWIN_BASH% --login -i
15.30 +'
15.31 +' ------------------------------------------------------------
15.32 +
15.33 +' setup the basic objects
15.34 +Dim cFSO
15.35 +Dim cShell
15.36 +Set cShell = CreateObject("WScript.Shell")
15.37 +Set cFSO = CreateObject("Scripting.FileSystemObject")
15.38 +
15.39 +' parse script location and cd into folder
15.40 +Dim sPath
15.41 +sPath = Wscript.ScriptFullName
15.42 +sPath = cFSO.GetAbsolutePathName(sPath)
15.43 +sPath = cFSO.GetParentFolderName(sPath)
15.44 +cShell.CurrentDirectory = sPath
15.45 +
15.46 +' locations of mintty and bash
15.47 +Dim sMinTTYPath
15.48 +Dim sBashPath
15.49 +sMinTTYPath = """" & sPath & "\bin\mintty" & """"
15.50 +sBashPath = """" & sPath & "\bin\bash" & """"
15.51 +
15.52 +' start a cygwin shell
15.53 +cShell.Run sMinTTYPath & " " & sBashPath & " --login -i", 1, false
16.1 --- /dev/null Thu Jan 01 00:00:00 1970 +0000
16.2 +++ b/OpenSecurity/cygwin64/content.txt Mon Dec 02 14:02:05 2013 +0100
16.3 @@ -0,0 +1,10 @@
16.4 +Content of Opensecurity/cygwin
16.5 +==============================
16.6 +
16.7 +This folder contains a complete Cygwin - 64 Bit with at least these packages installed:
16.8 +
16.9 + - bash
16.10 + - coreutils
16.11 + - openssh
16.12 + - xinit
16.13 + - genisofs
17.1 Binary file OpenSecurity/cygwin64/setup-x86_64.exe has changed
18.1 Binary file OpenSecurity/gfx/ait_logo.jpg has changed
19.1 Binary file OpenSecurity/gfx/ait_logo_no_claim.png has changed
20.1 Binary file OpenSecurity/gfx/bmvit_logo.jpg has changed
21.1 Binary file OpenSecurity/gfx/ffg_logo.jpg has changed
22.1 Binary file OpenSecurity/gfx/ikarus_logo.jpg has changed
23.1 Binary file OpenSecurity/gfx/kiras_logo.jpg has changed
24.1 Binary file OpenSecurity/gfx/linz_logo.jpg has changed
25.1 Binary file OpenSecurity/gfx/liqua_logo.jpg has changed
26.1 Binary file OpenSecurity/gfx/opensecurity.ico has changed
27.1 Binary file OpenSecurity/gfx/opensecurity_icon_64.png has changed
28.1 Binary file OpenSecurity/gfx/opensecurity_logo.jpg has changed
29.1 Binary file OpenSecurity/gfx/x-net_logo.jpg has changed
30.1 Binary file OpenSecurity/install/OpenSecurity.reg has changed
31.1 --- /dev/null Thu Jan 01 00:00:00 1970 +0000
31.2 +++ b/OpenSecurity/install/content.txt Mon Dec 02 14:02:05 2013 +0100
31.3 @@ -0,0 +1,15 @@
31.4 +Content of Opensecurity/install
31.5 +==============================
31.6 +
31.7 +Additonally to the files already present in this folder, we need:
31.8 +
31.9 +- PyQt4-4.10.3-gpl-Py2.7-Qt4.8.5-x32.exe
31.10 +- PyQt4-4.10.3-gpl-Py2.7-Qt4.8.5-x64.exe
31.11 + (--> http://www.riverbankcomputing.com/software/pyqt/download)
31.12 +
31.13 +- python-2.7.6.amd64.msi
31.14 +- python-2.7.6.msi
31.15 + (--> http://www.python.org/download)
31.16 +
31.17 +- VirtualBox-4.3.4-91027-Win.exe
31.18 + (--> https://virtualbox.org/wiki/Downloads)
32.1 --- /dev/null Thu Jan 01 00:00:00 1970 +0000
32.2 +++ b/OpenSecurity/install/web.py-0.37/PKG-INFO Mon Dec 02 14:02:05 2013 +0100
32.3 @@ -0,0 +1,10 @@
32.4 +Metadata-Version: 1.0
32.5 +Name: web.py
32.6 +Version: 0.37
32.7 +Summary: web.py: makes web apps
32.8 +Home-page: http://webpy.org/
32.9 +Author: Anand Chitipothu
32.10 +Author-email: anandology@gmail.com
32.11 +License: Public domain
32.12 +Description: Think about the ideal way to write a web app. Write the code to make it happen.
32.13 +Platform: any
33.1 --- /dev/null Thu Jan 01 00:00:00 1970 +0000
33.2 +++ b/OpenSecurity/install/web.py-0.37/build/lib/web/__init__.py Mon Dec 02 14:02:05 2013 +0100
33.3 @@ -0,0 +1,33 @@
33.4 +#!/usr/bin/env python
33.5 +"""web.py: makes web apps (http://webpy.org)"""
33.6 +
33.7 +from __future__ import generators
33.8 +
33.9 +__version__ = "0.37"
33.10 +__author__ = [
33.11 + "Aaron Swartz <me@aaronsw.com>",
33.12 + "Anand Chitipothu <anandology@gmail.com>"
33.13 +]
33.14 +__license__ = "public domain"
33.15 +__contributors__ = "see http://webpy.org/changes"
33.16 +
33.17 +import utils, db, net, wsgi, http, webapi, httpserver, debugerror
33.18 +import template, form
33.19 +
33.20 +import session
33.21 +
33.22 +from utils import *
33.23 +from db import *
33.24 +from net import *
33.25 +from wsgi import *
33.26 +from http import *
33.27 +from webapi import *
33.28 +from httpserver import *
33.29 +from debugerror import *
33.30 +from application import *
33.31 +from browser import *
33.32 +try:
33.33 + import webopenid as openid
33.34 +except ImportError:
33.35 + pass # requires openid module
33.36 +
34.1 --- /dev/null Thu Jan 01 00:00:00 1970 +0000
34.2 +++ b/OpenSecurity/install/web.py-0.37/build/lib/web/application.py Mon Dec 02 14:02:05 2013 +0100
34.3 @@ -0,0 +1,687 @@
34.4 +"""
34.5 +Web application
34.6 +(from web.py)
34.7 +"""
34.8 +import webapi as web
34.9 +import webapi, wsgi, utils
34.10 +import debugerror
34.11 +import httpserver
34.12 +
34.13 +from utils import lstrips, safeunicode
34.14 +import sys
34.15 +
34.16 +import urllib
34.17 +import traceback
34.18 +import itertools
34.19 +import os
34.20 +import types
34.21 +from exceptions import SystemExit
34.22 +
34.23 +try:
34.24 + import wsgiref.handlers
34.25 +except ImportError:
34.26 + pass # don't break people with old Pythons
34.27 +
34.28 +__all__ = [
34.29 + "application", "auto_application",
34.30 + "subdir_application", "subdomain_application",
34.31 + "loadhook", "unloadhook",
34.32 + "autodelegate"
34.33 +]
34.34 +
34.35 +class application:
34.36 + """
34.37 + Application to delegate requests based on path.
34.38 +
34.39 + >>> urls = ("/hello", "hello")
34.40 + >>> app = application(urls, globals())
34.41 + >>> class hello:
34.42 + ... def GET(self): return "hello"
34.43 + >>>
34.44 + >>> app.request("/hello").data
34.45 + 'hello'
34.46 + """
34.47 + def __init__(self, mapping=(), fvars={}, autoreload=None):
34.48 + if autoreload is None:
34.49 + autoreload = web.config.get('debug', False)
34.50 + self.init_mapping(mapping)
34.51 + self.fvars = fvars
34.52 + self.processors = []
34.53 +
34.54 + self.add_processor(loadhook(self._load))
34.55 + self.add_processor(unloadhook(self._unload))
34.56 +
34.57 + if autoreload:
34.58 + def main_module_name():
34.59 + mod = sys.modules['__main__']
34.60 + file = getattr(mod, '__file__', None) # make sure this works even from python interpreter
34.61 + return file and os.path.splitext(os.path.basename(file))[0]
34.62 +
34.63 + def modname(fvars):
34.64 + """find name of the module name from fvars."""
34.65 + file, name = fvars.get('__file__'), fvars.get('__name__')
34.66 + if file is None or name is None:
34.67 + return None
34.68 +
34.69 + if name == '__main__':
34.70 + # Since the __main__ module can't be reloaded, the module has
34.71 + # to be imported using its file name.
34.72 + name = main_module_name()
34.73 + return name
34.74 +
34.75 + mapping_name = utils.dictfind(fvars, mapping)
34.76 + module_name = modname(fvars)
34.77 +
34.78 + def reload_mapping():
34.79 + """loadhook to reload mapping and fvars."""
34.80 + mod = __import__(module_name, None, None, [''])
34.81 + mapping = getattr(mod, mapping_name, None)
34.82 + if mapping:
34.83 + self.fvars = mod.__dict__
34.84 + self.init_mapping(mapping)
34.85 +
34.86 + self.add_processor(loadhook(Reloader()))
34.87 + if mapping_name and module_name:
34.88 + self.add_processor(loadhook(reload_mapping))
34.89 +
34.90 + # load __main__ module usings its filename, so that it can be reloaded.
34.91 + if main_module_name() and '__main__' in sys.argv:
34.92 + try:
34.93 + __import__(main_module_name())
34.94 + except ImportError:
34.95 + pass
34.96 +
34.97 + def _load(self):
34.98 + web.ctx.app_stack.append(self)
34.99 +
34.100 + def _unload(self):
34.101 + web.ctx.app_stack = web.ctx.app_stack[:-1]
34.102 +
34.103 + if web.ctx.app_stack:
34.104 + # this is a sub-application, revert ctx to earlier state.
34.105 + oldctx = web.ctx.get('_oldctx')
34.106 + if oldctx:
34.107 + web.ctx.home = oldctx.home
34.108 + web.ctx.homepath = oldctx.homepath
34.109 + web.ctx.path = oldctx.path
34.110 + web.ctx.fullpath = oldctx.fullpath
34.111 +
34.112 + def _cleanup(self):
34.113 + # Threads can be recycled by WSGI servers.
34.114 + # Clearing up all thread-local state to avoid interefereing with subsequent requests.
34.115 + utils.ThreadedDict.clear_all()
34.116 +
34.117 + def init_mapping(self, mapping):
34.118 + self.mapping = list(utils.group(mapping, 2))
34.119 +
34.120 + def add_mapping(self, pattern, classname):
34.121 + self.mapping.append((pattern, classname))
34.122 +
34.123 + def add_processor(self, processor):
34.124 + """
34.125 + Adds a processor to the application.
34.126 +
34.127 + >>> urls = ("/(.*)", "echo")
34.128 + >>> app = application(urls, globals())
34.129 + >>> class echo:
34.130 + ... def GET(self, name): return name
34.131 + ...
34.132 + >>>
34.133 + >>> def hello(handler): return "hello, " + handler()
34.134 + ...
34.135 + >>> app.add_processor(hello)
34.136 + >>> app.request("/web.py").data
34.137 + 'hello, web.py'
34.138 + """
34.139 + self.processors.append(processor)
34.140 +
34.141 + def request(self, localpart='/', method='GET', data=None,
34.142 + host="0.0.0.0:8080", headers=None, https=False, **kw):
34.143 + """Makes request to this application for the specified path and method.
34.144 + Response will be a storage object with data, status and headers.
34.145 +
34.146 + >>> urls = ("/hello", "hello")
34.147 + >>> app = application(urls, globals())
34.148 + >>> class hello:
34.149 + ... def GET(self):
34.150 + ... web.header('Content-Type', 'text/plain')
34.151 + ... return "hello"
34.152 + ...
34.153 + >>> response = app.request("/hello")
34.154 + >>> response.data
34.155 + 'hello'
34.156 + >>> response.status
34.157 + '200 OK'
34.158 + >>> response.headers['Content-Type']
34.159 + 'text/plain'
34.160 +
34.161 + To use https, use https=True.
34.162 +
34.163 + >>> urls = ("/redirect", "redirect")
34.164 + >>> app = application(urls, globals())
34.165 + >>> class redirect:
34.166 + ... def GET(self): raise web.seeother("/foo")
34.167 + ...
34.168 + >>> response = app.request("/redirect")
34.169 + >>> response.headers['Location']
34.170 + 'http://0.0.0.0:8080/foo'
34.171 + >>> response = app.request("/redirect", https=True)
34.172 + >>> response.headers['Location']
34.173 + 'https://0.0.0.0:8080/foo'
34.174 +
34.175 + The headers argument specifies HTTP headers as a mapping object
34.176 + such as a dict.
34.177 +
34.178 + >>> urls = ('/ua', 'uaprinter')
34.179 + >>> class uaprinter:
34.180 + ... def GET(self):
34.181 + ... return 'your user-agent is ' + web.ctx.env['HTTP_USER_AGENT']
34.182 + ...
34.183 + >>> app = application(urls, globals())
34.184 + >>> app.request('/ua', headers = {
34.185 + ... 'User-Agent': 'a small jumping bean/1.0 (compatible)'
34.186 + ... }).data
34.187 + 'your user-agent is a small jumping bean/1.0 (compatible)'
34.188 +
34.189 + """
34.190 + path, maybe_query = urllib.splitquery(localpart)
34.191 + query = maybe_query or ""
34.192 +
34.193 + if 'env' in kw:
34.194 + env = kw['env']
34.195 + else:
34.196 + env = {}
34.197 + env = dict(env, HTTP_HOST=host, REQUEST_METHOD=method, PATH_INFO=path, QUERY_STRING=query, HTTPS=str(https))
34.198 + headers = headers or {}
34.199 +
34.200 + for k, v in headers.items():
34.201 + env['HTTP_' + k.upper().replace('-', '_')] = v
34.202 +
34.203 + if 'HTTP_CONTENT_LENGTH' in env:
34.204 + env['CONTENT_LENGTH'] = env.pop('HTTP_CONTENT_LENGTH')
34.205 +
34.206 + if 'HTTP_CONTENT_TYPE' in env:
34.207 + env['CONTENT_TYPE'] = env.pop('HTTP_CONTENT_TYPE')
34.208 +
34.209 + if method not in ["HEAD", "GET"]:
34.210 + data = data or ''
34.211 + import StringIO
34.212 + if isinstance(data, dict):
34.213 + q = urllib.urlencode(data)
34.214 + else:
34.215 + q = data
34.216 + env['wsgi.input'] = StringIO.StringIO(q)
34.217 + if not env.get('CONTENT_TYPE', '').lower().startswith('multipart/') and 'CONTENT_LENGTH' not in env:
34.218 + env['CONTENT_LENGTH'] = len(q)
34.219 + response = web.storage()
34.220 + def start_response(status, headers):
34.221 + response.status = status
34.222 + response.headers = dict(headers)
34.223 + response.header_items = headers
34.224 + response.data = "".join(self.wsgifunc()(env, start_response))
34.225 + return response
34.226 +
34.227 + def browser(self):
34.228 + import browser
34.229 + return browser.AppBrowser(self)
34.230 +
34.231 + def handle(self):
34.232 + fn, args = self._match(self.mapping, web.ctx.path)
34.233 + return self._delegate(fn, self.fvars, args)
34.234 +
34.235 + def handle_with_processors(self):
34.236 + def process(processors):
34.237 + try:
34.238 + if processors:
34.239 + p, processors = processors[0], processors[1:]
34.240 + return p(lambda: process(processors))
34.241 + else:
34.242 + return self.handle()
34.243 + except web.HTTPError:
34.244 + raise
34.245 + except (KeyboardInterrupt, SystemExit):
34.246 + raise
34.247 + except:
34.248 + print >> web.debug, traceback.format_exc()
34.249 + raise self.internalerror()
34.250 +
34.251 + # processors must be applied in the resvere order. (??)
34.252 + return process(self.processors)
34.253 +
34.254 + def wsgifunc(self, *middleware):
34.255 + """Returns a WSGI-compatible function for this application."""
34.256 + def peep(iterator):
34.257 + """Peeps into an iterator by doing an iteration
34.258 + and returns an equivalent iterator.
34.259 + """
34.260 + # wsgi requires the headers first
34.261 + # so we need to do an iteration
34.262 + # and save the result for later
34.263 + try:
34.264 + firstchunk = iterator.next()
34.265 + except StopIteration:
34.266 + firstchunk = ''
34.267 +
34.268 + return itertools.chain([firstchunk], iterator)
34.269 +
34.270 + def is_generator(x): return x and hasattr(x, 'next')
34.271 +
34.272 + def wsgi(env, start_resp):
34.273 + # clear threadlocal to avoid inteference of previous requests
34.274 + self._cleanup()
34.275 +
34.276 + self.load(env)
34.277 + try:
34.278 + # allow uppercase methods only
34.279 + if web.ctx.method.upper() != web.ctx.method:
34.280 + raise web.nomethod()
34.281 +
34.282 + result = self.handle_with_processors()
34.283 + if is_generator(result):
34.284 + result = peep(result)
34.285 + else:
34.286 + result = [result]
34.287 + except web.HTTPError, e:
34.288 + result = [e.data]
34.289 +
34.290 + result = web.safestr(iter(result))
34.291 +
34.292 + status, headers = web.ctx.status, web.ctx.headers
34.293 + start_resp(status, headers)
34.294 +
34.295 + def cleanup():
34.296 + self._cleanup()
34.297 + yield '' # force this function to be a generator
34.298 +
34.299 + return itertools.chain(result, cleanup())
34.300 +
34.301 + for m in middleware:
34.302 + wsgi = m(wsgi)
34.303 +
34.304 + return wsgi
34.305 +
34.306 + def run(self, *middleware):
34.307 + """
34.308 + Starts handling requests. If called in a CGI or FastCGI context, it will follow
34.309 + that protocol. If called from the command line, it will start an HTTP
34.310 + server on the port named in the first command line argument, or, if there
34.311 + is no argument, on port 8080.
34.312 +
34.313 + `middleware` is a list of WSGI middleware which is applied to the resulting WSGI
34.314 + function.
34.315 + """
34.316 + return wsgi.runwsgi(self.wsgifunc(*middleware))
34.317 +
34.318 + def stop(self):
34.319 + """Stops the http server started by run.
34.320 + """
34.321 + if httpserver.server:
34.322 + httpserver.server.stop()
34.323 + httpserver.server = None
34.324 +
34.325 + def cgirun(self, *middleware):
34.326 + """
34.327 + Return a CGI handler. This is mostly useful with Google App Engine.
34.328 + There you can just do:
34.329 +
34.330 + main = app.cgirun()
34.331 + """
34.332 + wsgiapp = self.wsgifunc(*middleware)
34.333 +
34.334 + try:
34.335 + from google.appengine.ext.webapp.util import run_wsgi_app
34.336 + return run_wsgi_app(wsgiapp)
34.337 + except ImportError:
34.338 + # we're not running from within Google App Engine
34.339 + return wsgiref.handlers.CGIHandler().run(wsgiapp)
34.340 +
34.341 + def load(self, env):
34.342 + """Initializes ctx using env."""
34.343 + ctx = web.ctx
34.344 + ctx.clear()
34.345 + ctx.status = '200 OK'
34.346 + ctx.headers = []
34.347 + ctx.output = ''
34.348 + ctx.environ = ctx.env = env
34.349 + ctx.host = env.get('HTTP_HOST')
34.350 +
34.351 + if env.get('wsgi.url_scheme') in ['http', 'https']:
34.352 + ctx.protocol = env['wsgi.url_scheme']
34.353 + elif env.get('HTTPS', '').lower() in ['on', 'true', '1']:
34.354 + ctx.protocol = 'https'
34.355 + else:
34.356 + ctx.protocol = 'http'
34.357 + ctx.homedomain = ctx.protocol + '://' + env.get('HTTP_HOST', '[unknown]')
34.358 + ctx.homepath = os.environ.get('REAL_SCRIPT_NAME', env.get('SCRIPT_NAME', ''))
34.359 + ctx.home = ctx.homedomain + ctx.homepath
34.360 + #@@ home is changed when the request is handled to a sub-application.
34.361 + #@@ but the real home is required for doing absolute redirects.
34.362 + ctx.realhome = ctx.home
34.363 + ctx.ip = env.get('REMOTE_ADDR')
34.364 + ctx.method = env.get('REQUEST_METHOD')
34.365 + ctx.path = env.get('PATH_INFO')
34.366 + # http://trac.lighttpd.net/trac/ticket/406 requires:
34.367 + if env.get('SERVER_SOFTWARE', '').startswith('lighttpd/'):
34.368 + ctx.path = lstrips(env.get('REQUEST_URI').split('?')[0], ctx.homepath)
34.369 + # Apache and CherryPy webservers unquote the url but lighttpd doesn't.
34.370 + # unquote explicitly for lighttpd to make ctx.path uniform across all servers.
34.371 + ctx.path = urllib.unquote(ctx.path)
34.372 +
34.373 + if env.get('QUERY_STRING'):
34.374 + ctx.query = '?' + env.get('QUERY_STRING', '')
34.375 + else:
34.376 + ctx.query = ''
34.377 +
34.378 + ctx.fullpath = ctx.path + ctx.query
34.379 +
34.380 + for k, v in ctx.iteritems():
34.381 + # convert all string values to unicode values and replace
34.382 + # malformed data with a suitable replacement marker.
34.383 + if isinstance(v, str):
34.384 + ctx[k] = v.decode('utf-8', 'replace')
34.385 +
34.386 + # status must always be str
34.387 + ctx.status = '200 OK'
34.388 +
34.389 + ctx.app_stack = []
34.390 +
34.391 + def _delegate(self, f, fvars, args=[]):
34.392 + def handle_class(cls):
34.393 + meth = web.ctx.method
34.394 + if meth == 'HEAD' and not hasattr(cls, meth):
34.395 + meth = 'GET'
34.396 + if not hasattr(cls, meth):
34.397 + raise web.nomethod(cls)
34.398 + tocall = getattr(cls(), meth)
34.399 + return tocall(*args)
34.400 +
34.401 + def is_class(o): return isinstance(o, (types.ClassType, type))
34.402 +
34.403 + if f is None:
34.404 + raise web.notfound()
34.405 + elif isinstance(f, application):
34.406 + return f.handle_with_processors()
34.407 + elif is_class(f):
34.408 + return handle_class(f)
34.409 + elif isinstance(f, basestring):
34.410 + if f.startswith('redirect '):
34.411 + url = f.split(' ', 1)[1]
34.412 + if web.ctx.method == "GET":
34.413 + x = web.ctx.env.get('QUERY_STRING', '')
34.414 + if x:
34.415 + url += '?' + x
34.416 + raise web.redirect(url)
34.417 + elif '.' in f:
34.418 + mod, cls = f.rsplit('.', 1)
34.419 + mod = __import__(mod, None, None, [''])
34.420 + cls = getattr(mod, cls)
34.421 + else:
34.422 + cls = fvars[f]
34.423 + return handle_class(cls)
34.424 + elif hasattr(f, '__call__'):
34.425 + return f()
34.426 + else:
34.427 + return web.notfound()
34.428 +
34.429 + def _match(self, mapping, value):
34.430 + for pat, what in mapping:
34.431 + if isinstance(what, application):
34.432 + if value.startswith(pat):
34.433 + f = lambda: self._delegate_sub_application(pat, what)
34.434 + return f, None
34.435 + else:
34.436 + continue
34.437 + elif isinstance(what, basestring):
34.438 + what, result = utils.re_subm('^' + pat + '$', what, value)
34.439 + else:
34.440 + result = utils.re_compile('^' + pat + '$').match(value)
34.441 +
34.442 + if result: # it's a match
34.443 + return what, [x for x in result.groups()]
34.444 + return None, None
34.445 +
34.446 + def _delegate_sub_application(self, dir, app):
34.447 + """Deletes request to sub application `app` rooted at the directory `dir`.
34.448 + The home, homepath, path and fullpath values in web.ctx are updated to mimic request
34.449 + to the subapp and are restored after it is handled.
34.450 +
34.451 + @@Any issues with when used with yield?
34.452 + """
34.453 + web.ctx._oldctx = web.storage(web.ctx)
34.454 + web.ctx.home += dir
34.455 + web.ctx.homepath += dir
34.456 + web.ctx.path = web.ctx.path[len(dir):]
34.457 + web.ctx.fullpath = web.ctx.fullpath[len(dir):]
34.458 + return app.handle_with_processors()
34.459 +
34.460 + def get_parent_app(self):
34.461 + if self in web.ctx.app_stack:
34.462 + index = web.ctx.app_stack.index(self)
34.463 + if index > 0:
34.464 + return web.ctx.app_stack[index-1]
34.465 +
34.466 + def notfound(self):
34.467 + """Returns HTTPError with '404 not found' message"""
34.468 + parent = self.get_parent_app()
34.469 + if parent:
34.470 + return parent.notfound()
34.471 + else:
34.472 + return web._NotFound()
34.473 +
34.474 + def internalerror(self):
34.475 + """Returns HTTPError with '500 internal error' message"""
34.476 + parent = self.get_parent_app()
34.477 + if parent:
34.478 + return parent.internalerror()
34.479 + elif web.config.get('debug'):
34.480 + import debugerror
34.481 + return debugerror.debugerror()
34.482 + else:
34.483 + return web._InternalError()
34.484 +
34.485 +class auto_application(application):
34.486 + """Application similar to `application` but urls are constructed
34.487 + automatiacally using metaclass.
34.488 +
34.489 + >>> app = auto_application()
34.490 + >>> class hello(app.page):
34.491 + ... def GET(self): return "hello, world"
34.492 + ...
34.493 + >>> class foo(app.page):
34.494 + ... path = '/foo/.*'
34.495 + ... def GET(self): return "foo"
34.496 + >>> app.request("/hello").data
34.497 + 'hello, world'
34.498 + >>> app.request('/foo/bar').data
34.499 + 'foo'
34.500 + """
34.501 + def __init__(self):
34.502 + application.__init__(self)
34.503 +
34.504 + class metapage(type):
34.505 + def __init__(klass, name, bases, attrs):
34.506 + type.__init__(klass, name, bases, attrs)
34.507 + path = attrs.get('path', '/' + name)
34.508 +
34.509 + # path can be specified as None to ignore that class
34.510 + # typically required to create a abstract base class.
34.511 + if path is not None:
34.512 + self.add_mapping(path, klass)
34.513 +
34.514 + class page:
34.515 + path = None
34.516 + __metaclass__ = metapage
34.517 +
34.518 + self.page = page
34.519 +
34.520 +# The application class already has the required functionality of subdir_application
34.521 +subdir_application = application
34.522 +
34.523 +class subdomain_application(application):
34.524 + """
34.525 + Application to delegate requests based on the host.
34.526 +
34.527 + >>> urls = ("/hello", "hello")
34.528 + >>> app = application(urls, globals())
34.529 + >>> class hello:
34.530 + ... def GET(self): return "hello"
34.531 + >>>
34.532 + >>> mapping = (r"hello\.example\.com", app)
34.533 + >>> app2 = subdomain_application(mapping)
34.534 + >>> app2.request("/hello", host="hello.example.com").data
34.535 + 'hello'
34.536 + >>> response = app2.request("/hello", host="something.example.com")
34.537 + >>> response.status
34.538 + '404 Not Found'
34.539 + >>> response.data
34.540 + 'not found'
34.541 + """
34.542 + def handle(self):
34.543 + host = web.ctx.host.split(':')[0] #strip port
34.544 + fn, args = self._match(self.mapping, host)
34.545 + return self._delegate(fn, self.fvars, args)
34.546 +
34.547 + def _match(self, mapping, value):
34.548 + for pat, what in mapping:
34.549 + if isinstance(what, basestring):
34.550 + what, result = utils.re_subm('^' + pat + '$', what, value)
34.551 + else:
34.552 + result = utils.re_compile('^' + pat + '$').match(value)
34.553 +
34.554 + if result: # it's a match
34.555 + return what, [x for x in result.groups()]
34.556 + return None, None
34.557 +
34.558 +def loadhook(h):
34.559 + """
34.560 + Converts a load hook into an application processor.
34.561 +
34.562 + >>> app = auto_application()
34.563 + >>> def f(): "something done before handling request"
34.564 + ...
34.565 + >>> app.add_processor(loadhook(f))
34.566 + """
34.567 + def processor(handler):
34.568 + h()
34.569 + return handler()
34.570 +
34.571 + return processor
34.572 +
34.573 +def unloadhook(h):
34.574 + """
34.575 + Converts an unload hook into an application processor.
34.576 +
34.577 + >>> app = auto_application()
34.578 + >>> def f(): "something done after handling request"
34.579 + ...
34.580 + >>> app.add_processor(unloadhook(f))
34.581 + """
34.582 + def processor(handler):
34.583 + try:
34.584 + result = handler()
34.585 + is_generator = result and hasattr(result, 'next')
34.586 + except:
34.587 + # run the hook even when handler raises some exception
34.588 + h()
34.589 + raise
34.590 +
34.591 + if is_generator:
34.592 + return wrap(result)
34.593 + else:
34.594 + h()
34.595 + return result
34.596 +
34.597 + def wrap(result):
34.598 + def next():
34.599 + try:
34.600 + return result.next()
34.601 + except:
34.602 + # call the hook at the and of iterator
34.603 + h()
34.604 + raise
34.605 +
34.606 + result = iter(result)
34.607 + while True:
34.608 + yield next()
34.609 +
34.610 + return processor
34.611 +
34.612 +def autodelegate(prefix=''):
34.613 + """
34.614 + Returns a method that takes one argument and calls the method named prefix+arg,
34.615 + calling `notfound()` if there isn't one. Example:
34.616 +
34.617 + urls = ('/prefs/(.*)', 'prefs')
34.618 +
34.619 + class prefs:
34.620 + GET = autodelegate('GET_')
34.621 + def GET_password(self): pass
34.622 + def GET_privacy(self): pass
34.623 +
34.624 + `GET_password` would get called for `/prefs/password` while `GET_privacy` for
34.625 + `GET_privacy` gets called for `/prefs/privacy`.
34.626 +
34.627 + If a user visits `/prefs/password/change` then `GET_password(self, '/change')`
34.628 + is called.
34.629 + """
34.630 + def internal(self, arg):
34.631 + if '/' in arg:
34.632 + first, rest = arg.split('/', 1)
34.633 + func = prefix + first
34.634 + args = ['/' + rest]
34.635 + else:
34.636 + func = prefix + arg
34.637 + args = []
34.638 +
34.639 + if hasattr(self, func):
34.640 + try:
34.641 + return getattr(self, func)(*args)
34.642 + except TypeError:
34.643 + raise web.notfound()
34.644 + else:
34.645 + raise web.notfound()
34.646 + return internal
34.647 +
34.648 +class Reloader:
34.649 + """Checks to see if any loaded modules have changed on disk and,
34.650 + if so, reloads them.
34.651 + """
34.652 +
34.653 + """File suffix of compiled modules."""
34.654 + if sys.platform.startswith('java'):
34.655 + SUFFIX = '$py.class'
34.656 + else:
34.657 + SUFFIX = '.pyc'
34.658 +
34.659 + def __init__(self):
34.660 + self.mtimes = {}
34.661 +
34.662 + def __call__(self):
34.663 + for mod in sys.modules.values():
34.664 + self.check(mod)
34.665 +
34.666 + def check(self, mod):
34.667 + # jython registers java packages as modules but they either
34.668 + # don't have a __file__ attribute or its value is None
34.669 + if not (mod and hasattr(mod, '__file__') and mod.__file__):
34.670 + return
34.671 +
34.672 + try:
34.673 + mtime = os.stat(mod.__file__).st_mtime
34.674 + except (OSError, IOError):
34.675 + return
34.676 + if mod.__file__.endswith(self.__class__.SUFFIX) and os.path.exists(mod.__file__[:-1]):
34.677 + mtime = max(os.stat(mod.__file__[:-1]).st_mtime, mtime)
34.678 +
34.679 + if mod not in self.mtimes:
34.680 + self.mtimes[mod] = mtime
34.681 + elif self.mtimes[mod] < mtime:
34.682 + try:
34.683 + reload(mod)
34.684 + self.mtimes[mod] = mtime
34.685 + except ImportError:
34.686 + pass
34.687 +
34.688 +if __name__ == "__main__":
34.689 + import doctest
34.690 + doctest.testmod()
35.1 --- /dev/null Thu Jan 01 00:00:00 1970 +0000
35.2 +++ b/OpenSecurity/install/web.py-0.37/build/lib/web/browser.py Mon Dec 02 14:02:05 2013 +0100
35.3 @@ -0,0 +1,236 @@
35.4 +"""Browser to test web applications.
35.5 +(from web.py)
35.6 +"""
35.7 +from utils import re_compile
35.8 +from net import htmlunquote
35.9 +
35.10 +import httplib, urllib, urllib2
35.11 +import copy
35.12 +from StringIO import StringIO
35.13 +
35.14 +DEBUG = False
35.15 +
35.16 +__all__ = [
35.17 + "BrowserError",
35.18 + "Browser", "AppBrowser",
35.19 + "AppHandler"
35.20 +]
35.21 +
35.22 +class BrowserError(Exception):
35.23 + pass
35.24 +
35.25 +class Browser:
35.26 + def __init__(self):
35.27 + import cookielib
35.28 + self.cookiejar = cookielib.CookieJar()
35.29 + self._cookie_processor = urllib2.HTTPCookieProcessor(self.cookiejar)
35.30 + self.form = None
35.31 +
35.32 + self.url = "http://0.0.0.0:8080/"
35.33 + self.path = "/"
35.34 +
35.35 + self.status = None
35.36 + self.data = None
35.37 + self._response = None
35.38 + self._forms = None
35.39 +
35.40 + def reset(self):
35.41 + """Clears all cookies and history."""
35.42 + self.cookiejar.clear()
35.43 +
35.44 + def build_opener(self):
35.45 + """Builds the opener using urllib2.build_opener.
35.46 + Subclasses can override this function to prodive custom openers.
35.47 + """
35.48 + return urllib2.build_opener()
35.49 +
35.50 + def do_request(self, req):
35.51 + if DEBUG:
35.52 + print 'requesting', req.get_method(), req.get_full_url()
35.53 + opener = self.build_opener()
35.54 + opener.add_handler(self._cookie_processor)
35.55 + try:
35.56 + self._response = opener.open(req)
35.57 + except urllib2.HTTPError, e:
35.58 + self._response = e
35.59 +
35.60 + self.url = self._response.geturl()
35.61 + self.path = urllib2.Request(self.url).get_selector()
35.62 + self.data = self._response.read()
35.63 + self.status = self._response.code
35.64 + self._forms = None
35.65 + self.form = None
35.66 + return self.get_response()
35.67 +
35.68 + def open(self, url, data=None, headers={}):
35.69 + """Opens the specified url."""
35.70 + url = urllib.basejoin(self.url, url)
35.71 + req = urllib2.Request(url, data, headers)
35.72 + return self.do_request(req)
35.73 +
35.74 + def show(self):
35.75 + """Opens the current page in real web browser."""
35.76 + f = open('page.html', 'w')
35.77 + f.write(self.data)
35.78 + f.close()
35.79 +
35.80 + import webbrowser, os
35.81 + url = 'file://' + os.path.abspath('page.html')
35.82 + webbrowser.open(url)
35.83 +
35.84 + def get_response(self):
35.85 + """Returns a copy of the current response."""
35.86 + return urllib.addinfourl(StringIO(self.data), self._response.info(), self._response.geturl())
35.87 +
35.88 + def get_soup(self):
35.89 + """Returns beautiful soup of the current document."""
35.90 + import BeautifulSoup
35.91 + return BeautifulSoup.BeautifulSoup(self.data)
35.92 +
35.93 + def get_text(self, e=None):
35.94 + """Returns content of e or the current document as plain text."""
35.95 + e = e or self.get_soup()
35.96 + return ''.join([htmlunquote(c) for c in e.recursiveChildGenerator() if isinstance(c, unicode)])
35.97 +
35.98 + def _get_links(self):
35.99 + soup = self.get_soup()
35.100 + return [a for a in soup.findAll(name='a')]
35.101 +
35.102 + def get_links(self, text=None, text_regex=None, url=None, url_regex=None, predicate=None):
35.103 + """Returns all links in the document."""
35.104 + return self._filter_links(self._get_links(),
35.105 + text=text, text_regex=text_regex, url=url, url_regex=url_regex, predicate=predicate)
35.106 +
35.107 + def follow_link(self, link=None, text=None, text_regex=None, url=None, url_regex=None, predicate=None):
35.108 + if link is None:
35.109 + links = self._filter_links(self.get_links(),
35.110 + text=text, text_regex=text_regex, url=url, url_regex=url_regex, predicate=predicate)
35.111 + link = links and links[0]
35.112 +
35.113 + if link:
35.114 + return self.open(link['href'])
35.115 + else:
35.116 + raise BrowserError("No link found")
35.117 +
35.118 + def find_link(self, text=None, text_regex=None, url=None, url_regex=None, predicate=None):
35.119 + links = self._filter_links(self.get_links(),
35.120 + text=text, text_regex=text_regex, url=url, url_regex=url_regex, predicate=predicate)
35.121 + return links and links[0] or None
35.122 +
35.123 + def _filter_links(self, links,
35.124 + text=None, text_regex=None,
35.125 + url=None, url_regex=None,
35.126 + predicate=None):
35.127 + predicates = []
35.128 + if text is not None:
35.129 + predicates.append(lambda link: link.string == text)
35.130 + if text_regex is not None:
35.131 + predicates.append(lambda link: re_compile(text_regex).search(link.string or ''))
35.132 + if url is not None:
35.133 + predicates.append(lambda link: link.get('href') == url)
35.134 + if url_regex is not None:
35.135 + predicates.append(lambda link: re_compile(url_regex).search(link.get('href', '')))
35.136 + if predicate:
35.137 + predicate.append(predicate)
35.138 +
35.139 + def f(link):
35.140 + for p in predicates:
35.141 + if not p(link):
35.142 + return False
35.143 + return True
35.144 +
35.145 + return [link for link in links if f(link)]
35.146 +
35.147 + def get_forms(self):
35.148 + """Returns all forms in the current document.
35.149 + The returned form objects implement the ClientForm.HTMLForm interface.
35.150 + """
35.151 + if self._forms is None:
35.152 + import ClientForm
35.153 + self._forms = ClientForm.ParseResponse(self.get_response(), backwards_compat=False)
35.154 + return self._forms
35.155 +
35.156 + def select_form(self, name=None, predicate=None, index=0):
35.157 + """Selects the specified form."""
35.158 + forms = self.get_forms()
35.159 +
35.160 + if name is not None:
35.161 + forms = [f for f in forms if f.name == name]
35.162 + if predicate:
35.163 + forms = [f for f in forms if predicate(f)]
35.164 +
35.165 + if forms:
35.166 + self.form = forms[index]
35.167 + return self.form
35.168 + else:
35.169 + raise BrowserError("No form selected.")
35.170 +
35.171 + def submit(self, **kw):
35.172 + """submits the currently selected form."""
35.173 + if self.form is None:
35.174 + raise BrowserError("No form selected.")
35.175 + req = self.form.click(**kw)
35.176 + return self.do_request(req)
35.177 +
35.178 + def __getitem__(self, key):
35.179 + return self.form[key]
35.180 +
35.181 + def __setitem__(self, key, value):
35.182 + self.form[key] = value
35.183 +
35.184 +class AppBrowser(Browser):
35.185 + """Browser interface to test web.py apps.
35.186 +
35.187 + b = AppBrowser(app)
35.188 + b.open('/')
35.189 + b.follow_link(text='Login')
35.190 +
35.191 + b.select_form(name='login')
35.192 + b['username'] = 'joe'
35.193 + b['password'] = 'secret'
35.194 + b.submit()
35.195 +
35.196 + assert b.path == '/'
35.197 + assert 'Welcome joe' in b.get_text()
35.198 + """
35.199 + def __init__(self, app):
35.200 + Browser.__init__(self)
35.201 + self.app = app
35.202 +
35.203 + def build_opener(self):
35.204 + return urllib2.build_opener(AppHandler(self.app))
35.205 +
35.206 +class AppHandler(urllib2.HTTPHandler):
35.207 + """urllib2 handler to handle requests using web.py application."""
35.208 + handler_order = 100
35.209 +
35.210 + def __init__(self, app):
35.211 + self.app = app
35.212 +
35.213 + def http_open(self, req):
35.214 + result = self.app.request(
35.215 + localpart=req.get_selector(),
35.216 + method=req.get_method(),
35.217 + host=req.get_host(),
35.218 + data=req.get_data(),
35.219 + headers=dict(req.header_items()),
35.220 + https=req.get_type() == "https"
35.221 + )
35.222 + return self._make_response(result, req.get_full_url())
35.223 +
35.224 + def https_open(self, req):
35.225 + return self.http_open(req)
35.226 +
35.227 + try:
35.228 + https_request = urllib2.HTTPHandler.do_request_
35.229 + except AttributeError:
35.230 + # for python 2.3
35.231 + pass
35.232 +
35.233 + def _make_response(self, result, url):
35.234 + data = "\r\n".join(["%s: %s" % (k, v) for k, v in result.header_items])
35.235 + headers = httplib.HTTPMessage(StringIO(data))
35.236 + response = urllib.addinfourl(StringIO(result.data), headers, url)
35.237 + code, msg = result.status.split(None, 1)
35.238 + response.code, response.msg = int(code), msg
35.239 + return response
36.1 --- /dev/null Thu Jan 01 00:00:00 1970 +0000
36.2 +++ b/OpenSecurity/install/web.py-0.37/build/lib/web/contrib/template.py Mon Dec 02 14:02:05 2013 +0100
36.3 @@ -0,0 +1,131 @@
36.4 +"""
36.5 +Interface to various templating engines.
36.6 +"""
36.7 +import os.path
36.8 +
36.9 +__all__ = [
36.10 + "render_cheetah", "render_genshi", "render_mako",
36.11 + "cache",
36.12 +]
36.13 +
36.14 +class render_cheetah:
36.15 + """Rendering interface to Cheetah Templates.
36.16 +
36.17 + Example:
36.18 +
36.19 + render = render_cheetah('templates')
36.20 + render.hello(name="cheetah")
36.21 + """
36.22 + def __init__(self, path):
36.23 + # give error if Chetah is not installed
36.24 + from Cheetah.Template import Template
36.25 + self.path = path
36.26 +
36.27 + def __getattr__(self, name):
36.28 + from Cheetah.Template import Template
36.29 + path = os.path.join(self.path, name + ".html")
36.30 +
36.31 + def template(**kw):
36.32 + t = Template(file=path, searchList=[kw])
36.33 + return t.respond()
36.34 +
36.35 + return template
36.36 +
36.37 +class render_genshi:
36.38 + """Rendering interface genshi templates.
36.39 + Example:
36.40 +
36.41 + for xml/html templates.
36.42 +
36.43 + render = render_genshi(['templates/'])
36.44 + render.hello(name='genshi')
36.45 +
36.46 + For text templates:
36.47 +
36.48 + render = render_genshi(['templates/'], type='text')
36.49 + render.hello(name='genshi')
36.50 + """
36.51 +
36.52 + def __init__(self, *a, **kwargs):
36.53 + from genshi.template import TemplateLoader
36.54 +
36.55 + self._type = kwargs.pop('type', None)
36.56 + self._loader = TemplateLoader(*a, **kwargs)
36.57 +
36.58 + def __getattr__(self, name):
36.59 + # Assuming all templates are html
36.60 + path = name + ".html"
36.61 +
36.62 + if self._type == "text":
36.63 + from genshi.template import TextTemplate
36.64 + cls = TextTemplate
36.65 + type = "text"
36.66 + else:
36.67 + cls = None
36.68 + type = None
36.69 +
36.70 + t = self._loader.load(path, cls=cls)
36.71 + def template(**kw):
36.72 + stream = t.generate(**kw)
36.73 + if type:
36.74 + return stream.render(type)
36.75 + else:
36.76 + return stream.render()
36.77 + return template
36.78 +
36.79 +class render_jinja:
36.80 + """Rendering interface to Jinja2 Templates
36.81 +
36.82 + Example:
36.83 +
36.84 + render= render_jinja('templates')
36.85 + render.hello(name='jinja2')
36.86 + """
36.87 + def __init__(self, *a, **kwargs):
36.88 + extensions = kwargs.pop('extensions', [])
36.89 + globals = kwargs.pop('globals', {})
36.90 +
36.91 + from jinja2 import Environment,FileSystemLoader
36.92 + self._lookup = Environment(loader=FileSystemLoader(*a, **kwargs), extensions=extensions)
36.93 + self._lookup.globals.update(globals)
36.94 +
36.95 + def __getattr__(self, name):
36.96 + # Assuming all templates end with .html
36.97 + path = name + '.html'
36.98 + t = self._lookup.get_template(path)
36.99 + return t.render
36.100 +
36.101 +class render_mako:
36.102 + """Rendering interface to Mako Templates.
36.103 +
36.104 + Example:
36.105 +
36.106 + render = render_mako(directories=['templates'])
36.107 + render.hello(name="mako")
36.108 + """
36.109 + def __init__(self, *a, **kwargs):
36.110 + from mako.lookup import TemplateLookup
36.111 + self._lookup = TemplateLookup(*a, **kwargs)
36.112 +
36.113 + def __getattr__(self, name):
36.114 + # Assuming all templates are html
36.115 + path = name + ".html"
36.116 + t = self._lookup.get_template(path)
36.117 + return t.render
36.118 +
36.119 +class cache:
36.120 + """Cache for any rendering interface.
36.121 +
36.122 + Example:
36.123 +
36.124 + render = cache(render_cheetah("templates/"))
36.125 + render.hello(name='cache')
36.126 + """
36.127 + def __init__(self, render):
36.128 + self._render = render
36.129 + self._cache = {}
36.130 +
36.131 + def __getattr__(self, name):
36.132 + if name not in self._cache:
36.133 + self._cache[name] = getattr(self._render, name)
36.134 + return self._cache[name]
37.1 --- /dev/null Thu Jan 01 00:00:00 1970 +0000
37.2 +++ b/OpenSecurity/install/web.py-0.37/build/lib/web/db.py Mon Dec 02 14:02:05 2013 +0100
37.3 @@ -0,0 +1,1237 @@
37.4 +"""
37.5 +Database API
37.6 +(part of web.py)
37.7 +"""
37.8 +
37.9 +__all__ = [
37.10 + "UnknownParamstyle", "UnknownDB", "TransactionError",
37.11 + "sqllist", "sqlors", "reparam", "sqlquote",
37.12 + "SQLQuery", "SQLParam", "sqlparam",
37.13 + "SQLLiteral", "sqlliteral",
37.14 + "database", 'DB',
37.15 +]
37.16 +
37.17 +import time
37.18 +try:
37.19 + import datetime
37.20 +except ImportError:
37.21 + datetime = None
37.22 +
37.23 +try: set
37.24 +except NameError:
37.25 + from sets import Set as set
37.26 +
37.27 +from utils import threadeddict, storage, iters, iterbetter, safestr, safeunicode
37.28 +
37.29 +try:
37.30 + # db module can work independent of web.py
37.31 + from webapi import debug, config
37.32 +except:
37.33 + import sys
37.34 + debug = sys.stderr
37.35 + config = storage()
37.36 +
37.37 +class UnknownDB(Exception):
37.38 + """raised for unsupported dbms"""
37.39 + pass
37.40 +
37.41 +class _ItplError(ValueError):
37.42 + def __init__(self, text, pos):
37.43 + ValueError.__init__(self)
37.44 + self.text = text
37.45 + self.pos = pos
37.46 + def __str__(self):
37.47 + return "unfinished expression in %s at char %d" % (
37.48 + repr(self.text), self.pos)
37.49 +
37.50 +class TransactionError(Exception): pass
37.51 +
37.52 +class UnknownParamstyle(Exception):
37.53 + """
37.54 + raised for unsupported db paramstyles
37.55 +
37.56 + (currently supported: qmark, numeric, format, pyformat)
37.57 + """
37.58 + pass
37.59 +
37.60 +class SQLParam(object):
37.61 + """
37.62 + Parameter in SQLQuery.
37.63 +
37.64 + >>> q = SQLQuery(["SELECT * FROM test WHERE name=", SQLParam("joe")])
37.65 + >>> q
37.66 + <sql: "SELECT * FROM test WHERE name='joe'">
37.67 + >>> q.query()
37.68 + 'SELECT * FROM test WHERE name=%s'
37.69 + >>> q.values()
37.70 + ['joe']
37.71 + """
37.72 + __slots__ = ["value"]
37.73 +
37.74 + def __init__(self, value):
37.75 + self.value = value
37.76 +
37.77 + def get_marker(self, paramstyle='pyformat'):
37.78 + if paramstyle == 'qmark':
37.79 + return '?'
37.80 + elif paramstyle == 'numeric':
37.81 + return ':1'
37.82 + elif paramstyle is None or paramstyle in ['format', 'pyformat']:
37.83 + return '%s'
37.84 + raise UnknownParamstyle, paramstyle
37.85 +
37.86 + def sqlquery(self):
37.87 + return SQLQuery([self])
37.88 +
37.89 + def __add__(self, other):
37.90 + return self.sqlquery() + other
37.91 +
37.92 + def __radd__(self, other):
37.93 + return other + self.sqlquery()
37.94 +
37.95 + def __str__(self):
37.96 + return str(self.value)
37.97 +
37.98 + def __repr__(self):
37.99 + return '<param: %s>' % repr(self.value)
37.100 +
37.101 +sqlparam = SQLParam
37.102 +
37.103 +class SQLQuery(object):
37.104 + """
37.105 + You can pass this sort of thing as a clause in any db function.
37.106 + Otherwise, you can pass a dictionary to the keyword argument `vars`
37.107 + and the function will call reparam for you.
37.108 +
37.109 + Internally, consists of `items`, which is a list of strings and
37.110 + SQLParams, which get concatenated to produce the actual query.
37.111 + """
37.112 + __slots__ = ["items"]
37.113 +
37.114 + # tested in sqlquote's docstring
37.115 + def __init__(self, items=None):
37.116 + r"""Creates a new SQLQuery.
37.117 +
37.118 + >>> SQLQuery("x")
37.119 + <sql: 'x'>
37.120 + >>> q = SQLQuery(['SELECT * FROM ', 'test', ' WHERE x=', SQLParam(1)])
37.121 + >>> q
37.122 + <sql: 'SELECT * FROM test WHERE x=1'>
37.123 + >>> q.query(), q.values()
37.124 + ('SELECT * FROM test WHERE x=%s', [1])
37.125 + >>> SQLQuery(SQLParam(1))
37.126 + <sql: '1'>
37.127 + """
37.128 + if items is None:
37.129 + self.items = []
37.130 + elif isinstance(items, list):
37.131 + self.items = items
37.132 + elif isinstance(items, SQLParam):
37.133 + self.items = [items]
37.134 + elif isinstance(items, SQLQuery):
37.135 + self.items = list(items.items)
37.136 + else:
37.137 + self.items = [items]
37.138 +
37.139 + # Take care of SQLLiterals
37.140 + for i, item in enumerate(self.items):
37.141 + if isinstance(item, SQLParam) and isinstance(item.value, SQLLiteral):
37.142 + self.items[i] = item.value.v
37.143 +
37.144 + def append(self, value):
37.145 + self.items.append(value)
37.146 +
37.147 + def __add__(self, other):
37.148 + if isinstance(other, basestring):
37.149 + items = [other]
37.150 + elif isinstance(other, SQLQuery):
37.151 + items = other.items
37.152 + else:
37.153 + return NotImplemented
37.154 + return SQLQuery(self.items + items)
37.155 +
37.156 + def __radd__(self, other):
37.157 + if isinstance(other, basestring):
37.158 + items = [other]
37.159 + else:
37.160 + return NotImplemented
37.161 +
37.162 + return SQLQuery(items + self.items)
37.163 +
37.164 + def __iadd__(self, other):
37.165 + if isinstance(other, (basestring, SQLParam)):
37.166 + self.items.append(other)
37.167 + elif isinstance(other, SQLQuery):
37.168 + self.items.extend(other.items)
37.169 + else:
37.170 + return NotImplemented
37.171 + return self
37.172 +
37.173 + def __len__(self):
37.174 + return len(self.query())
37.175 +
37.176 + def query(self, paramstyle=None):
37.177 + """
37.178 + Returns the query part of the sql query.
37.179 + >>> q = SQLQuery(["SELECT * FROM test WHERE name=", SQLParam('joe')])
37.180 + >>> q.query()
37.181 + 'SELECT * FROM test WHERE name=%s'
37.182 + >>> q.query(paramstyle='qmark')
37.183 + 'SELECT * FROM test WHERE name=?'
37.184 + """
37.185 + s = []
37.186 + for x in self.items:
37.187 + if isinstance(x, SQLParam):
37.188 + x = x.get_marker(paramstyle)
37.189 + s.append(safestr(x))
37.190 + else:
37.191 + x = safestr(x)
37.192 + # automatically escape % characters in the query
37.193 + # For backward compatability, ignore escaping when the query looks already escaped
37.194 + if paramstyle in ['format', 'pyformat']:
37.195 + if '%' in x and '%%' not in x:
37.196 + x = x.replace('%', '%%')
37.197 + s.append(x)
37.198 + return "".join(s)
37.199 +
37.200 + def values(self):
37.201 + """
37.202 + Returns the values of the parameters used in the sql query.
37.203 + >>> q = SQLQuery(["SELECT * FROM test WHERE name=", SQLParam('joe')])
37.204 + >>> q.values()
37.205 + ['joe']
37.206 + """
37.207 + return [i.value for i in self.items if isinstance(i, SQLParam)]
37.208 +
37.209 + def join(items, sep=' ', prefix=None, suffix=None, target=None):
37.210 + """
37.211 + Joins multiple queries.
37.212 +
37.213 + >>> SQLQuery.join(['a', 'b'], ', ')
37.214 + <sql: 'a, b'>
37.215 +
37.216 + Optinally, prefix and suffix arguments can be provided.
37.217 +
37.218 + >>> SQLQuery.join(['a', 'b'], ', ', prefix='(', suffix=')')
37.219 + <sql: '(a, b)'>
37.220 +
37.221 + If target argument is provided, the items are appended to target instead of creating a new SQLQuery.
37.222 + """
37.223 + if target is None:
37.224 + target = SQLQuery()
37.225 +
37.226 + target_items = target.items
37.227 +
37.228 + if prefix:
37.229 + target_items.append(prefix)
37.230 +
37.231 + for i, item in enumerate(items):
37.232 + if i != 0:
37.233 + target_items.append(sep)
37.234 + if isinstance(item, SQLQuery):
37.235 + target_items.extend(item.items)
37.236 + else:
37.237 + target_items.append(item)
37.238 +
37.239 + if suffix:
37.240 + target_items.append(suffix)
37.241 + return target
37.242 +
37.243 + join = staticmethod(join)
37.244 +
37.245 + def _str(self):
37.246 + try:
37.247 + return self.query() % tuple([sqlify(x) for x in self.values()])
37.248 + except (ValueError, TypeError):
37.249 + return self.query()
37.250 +
37.251 + def __str__(self):
37.252 + return safestr(self._str())
37.253 +
37.254 + def __unicode__(self):
37.255 + return safeunicode(self._str())
37.256 +
37.257 + def __repr__(self):
37.258 + return '<sql: %s>' % repr(str(self))
37.259 +
37.260 +class SQLLiteral:
37.261 + """
37.262 + Protects a string from `sqlquote`.
37.263 +
37.264 + >>> sqlquote('NOW()')
37.265 + <sql: "'NOW()'">
37.266 + >>> sqlquote(SQLLiteral('NOW()'))
37.267 + <sql: 'NOW()'>
37.268 + """
37.269 + def __init__(self, v):
37.270 + self.v = v
37.271 +
37.272 + def __repr__(self):
37.273 + return self.v
37.274 +
37.275 +sqlliteral = SQLLiteral
37.276 +
37.277 +def _sqllist(values):
37.278 + """
37.279 + >>> _sqllist([1, 2, 3])
37.280 + <sql: '(1, 2, 3)'>
37.281 + """
37.282 + items = []
37.283 + items.append('(')
37.284 + for i, v in enumerate(values):
37.285 + if i != 0:
37.286 + items.append(', ')
37.287 + items.append(sqlparam(v))
37.288 + items.append(')')
37.289 + return SQLQuery(items)
37.290 +
37.291 +def reparam(string_, dictionary):
37.292 + """
37.293 + Takes a string and a dictionary and interpolates the string
37.294 + using values from the dictionary. Returns an `SQLQuery` for the result.
37.295 +
37.296 + >>> reparam("s = $s", dict(s=True))
37.297 + <sql: "s = 't'">
37.298 + >>> reparam("s IN $s", dict(s=[1, 2]))
37.299 + <sql: 's IN (1, 2)'>
37.300 + """
37.301 + dictionary = dictionary.copy() # eval mucks with it
37.302 + vals = []
37.303 + result = []
37.304 + for live, chunk in _interpolate(string_):
37.305 + if live:
37.306 + v = eval(chunk, dictionary)
37.307 + result.append(sqlquote(v))
37.308 + else:
37.309 + result.append(chunk)
37.310 + return SQLQuery.join(result, '')
37.311 +
37.312 +def sqlify(obj):
37.313 + """
37.314 + converts `obj` to its proper SQL version
37.315 +
37.316 + >>> sqlify(None)
37.317 + 'NULL'
37.318 + >>> sqlify(True)
37.319 + "'t'"
37.320 + >>> sqlify(3)
37.321 + '3'
37.322 + """
37.323 + # because `1 == True and hash(1) == hash(True)`
37.324 + # we have to do this the hard way...
37.325 +
37.326 + if obj is None:
37.327 + return 'NULL'
37.328 + elif obj is True:
37.329 + return "'t'"
37.330 + elif obj is False:
37.331 + return "'f'"
37.332 + elif datetime and isinstance(obj, datetime.datetime):
37.333 + return repr(obj.isoformat())
37.334 + else:
37.335 + if isinstance(obj, unicode): obj = obj.encode('utf8')
37.336 + return repr(obj)
37.337 +
37.338 +def sqllist(lst):
37.339 + """
37.340 + Converts the arguments for use in something like a WHERE clause.
37.341 +
37.342 + >>> sqllist(['a', 'b'])
37.343 + 'a, b'
37.344 + >>> sqllist('a')
37.345 + 'a'
37.346 + >>> sqllist(u'abc')
37.347 + u'abc'
37.348 + """
37.349 + if isinstance(lst, basestring):
37.350 + return lst
37.351 + else:
37.352 + return ', '.join(lst)
37.353 +
37.354 +def sqlors(left, lst):
37.355 + """
37.356 + `left is a SQL clause like `tablename.arg = `
37.357 + and `lst` is a list of values. Returns a reparam-style
37.358 + pair featuring the SQL that ORs together the clause
37.359 + for each item in the lst.
37.360 +
37.361 + >>> sqlors('foo = ', [])
37.362 + <sql: '1=2'>
37.363 + >>> sqlors('foo = ', [1])
37.364 + <sql: 'foo = 1'>
37.365 + >>> sqlors('foo = ', 1)
37.366 + <sql: 'foo = 1'>
37.367 + >>> sqlors('foo = ', [1,2,3])
37.368 + <sql: '(foo = 1 OR foo = 2 OR foo = 3 OR 1=2)'>
37.369 + """
37.370 + if isinstance(lst, iters):
37.371 + lst = list(lst)
37.372 + ln = len(lst)
37.373 + if ln == 0:
37.374 + return SQLQuery("1=2")
37.375 + if ln == 1:
37.376 + lst = lst[0]
37.377 +
37.378 + if isinstance(lst, iters):
37.379 + return SQLQuery(['('] +
37.380 + sum([[left, sqlparam(x), ' OR '] for x in lst], []) +
37.381 + ['1=2)']
37.382 + )
37.383 + else:
37.384 + return left + sqlparam(lst)
37.385 +
37.386 +def sqlwhere(dictionary, grouping=' AND '):
37.387 + """
37.388 + Converts a `dictionary` to an SQL WHERE clause `SQLQuery`.
37.389 +
37.390 + >>> sqlwhere({'cust_id': 2, 'order_id':3})
37.391 + <sql: 'order_id = 3 AND cust_id = 2'>
37.392 + >>> sqlwhere({'cust_id': 2, 'order_id':3}, grouping=', ')
37.393 + <sql: 'order_id = 3, cust_id = 2'>
37.394 + >>> sqlwhere({'a': 'a', 'b': 'b'}).query()
37.395 + 'a = %s AND b = %s'
37.396 + """
37.397 + return SQLQuery.join([k + ' = ' + sqlparam(v) for k, v in dictionary.items()], grouping)
37.398 +
37.399 +def sqlquote(a):
37.400 + """
37.401 + Ensures `a` is quoted properly for use in a SQL query.
37.402 +
37.403 + >>> 'WHERE x = ' + sqlquote(True) + ' AND y = ' + sqlquote(3)
37.404 + <sql: "WHERE x = 't' AND y = 3">
37.405 + >>> 'WHERE x = ' + sqlquote(True) + ' AND y IN ' + sqlquote([2, 3])
37.406 + <sql: "WHERE x = 't' AND y IN (2, 3)">
37.407 + """
37.408 + if isinstance(a, list):
37.409 + return _sqllist(a)
37.410 + else:
37.411 + return sqlparam(a).sqlquery()
37.412 +
37.413 +class Transaction:
37.414 + """Database transaction."""
37.415 + def __init__(self, ctx):
37.416 + self.ctx = ctx
37.417 + self.transaction_count = transaction_count = len(ctx.transactions)
37.418 +
37.419 + class transaction_engine:
37.420 + """Transaction Engine used in top level transactions."""
37.421 + def do_transact(self):
37.422 + ctx.commit(unload=False)
37.423 +
37.424 + def do_commit(self):
37.425 + ctx.commit()
37.426 +
37.427 + def do_rollback(self):
37.428 + ctx.rollback()
37.429 +
37.430 + class subtransaction_engine:
37.431 + """Transaction Engine used in sub transactions."""
37.432 + def query(self, q):
37.433 + db_cursor = ctx.db.cursor()
37.434 + ctx.db_execute(db_cursor, SQLQuery(q % transaction_count))
37.435 +
37.436 + def do_transact(self):
37.437 + self.query('SAVEPOINT webpy_sp_%s')
37.438 +
37.439 + def do_commit(self):
37.440 + self.query('RELEASE SAVEPOINT webpy_sp_%s')
37.441 +
37.442 + def do_rollback(self):
37.443 + self.query('ROLLBACK TO SAVEPOINT webpy_sp_%s')
37.444 +
37.445 + class dummy_engine:
37.446 + """Transaction Engine used instead of subtransaction_engine
37.447 + when sub transactions are not supported."""
37.448 + do_transact = do_commit = do_rollback = lambda self: None
37.449 +
37.450 + if self.transaction_count:
37.451 + # nested transactions are not supported in some databases
37.452 + if self.ctx.get('ignore_nested_transactions'):
37.453 + self.engine = dummy_engine()
37.454 + else:
37.455 + self.engine = subtransaction_engine()
37.456 + else:
37.457 + self.engine = transaction_engine()
37.458 +
37.459 + self.engine.do_transact()
37.460 + self.ctx.transactions.append(self)
37.461 +
37.462 + def __enter__(self):
37.463 + return self
37.464 +
37.465 + def __exit__(self, exctype, excvalue, traceback):
37.466 + if exctype is not None:
37.467 + self.rollback()
37.468 + else:
37.469 + self.commit()
37.470 +
37.471 + def commit(self):
37.472 + if len(self.ctx.transactions) > self.transaction_count:
37.473 + self.engine.do_commit()
37.474 + self.ctx.transactions = self.ctx.transactions[:self.transaction_count]
37.475 +
37.476 + def rollback(self):
37.477 + if len(self.ctx.transactions) > self.transaction_count:
37.478 + self.engine.do_rollback()
37.479 + self.ctx.transactions = self.ctx.transactions[:self.transaction_count]
37.480 +
37.481 +class DB:
37.482 + """Database"""
37.483 + def __init__(self, db_module, keywords):
37.484 + """Creates a database.
37.485 + """
37.486 + # some DB implementaions take optional paramater `driver` to use a specific driver modue
37.487 + # but it should not be passed to connect
37.488 + keywords.pop('driver', None)
37.489 +
37.490 + self.db_module = db_module
37.491 + self.keywords = keywords
37.492 +
37.493 + self._ctx = threadeddict()
37.494 + # flag to enable/disable printing queries
37.495 + self.printing = config.get('debug_sql', config.get('debug', False))
37.496 + self.supports_multiple_insert = False
37.497 +
37.498 + try:
37.499 + import DBUtils
37.500 + # enable pooling if DBUtils module is available.
37.501 + self.has_pooling = True
37.502 + except ImportError:
37.503 + self.has_pooling = False
37.504 +
37.505 + # Pooling can be disabled by passing pooling=False in the keywords.
37.506 + self.has_pooling = self.keywords.pop('pooling', True) and self.has_pooling
37.507 +
37.508 + def _getctx(self):
37.509 + if not self._ctx.get('db'):
37.510 + self._load_context(self._ctx)
37.511 + return self._ctx
37.512 + ctx = property(_getctx)
37.513 +
37.514 + def _load_context(self, ctx):
37.515 + ctx.dbq_count = 0
37.516 + ctx.transactions = [] # stack of transactions
37.517 +
37.518 + if self.has_pooling:
37.519 + ctx.db = self._connect_with_pooling(self.keywords)
37.520 + else:
37.521 + ctx.db = self._connect(self.keywords)
37.522 + ctx.db_execute = self._db_execute
37.523 +
37.524 + if not hasattr(ctx.db, 'commit'):
37.525 + ctx.db.commit = lambda: None
37.526 +
37.527 + if not hasattr(ctx.db, 'rollback'):
37.528 + ctx.db.rollback = lambda: None
37.529 +
37.530 + def commit(unload=True):
37.531 + # do db commit and release the connection if pooling is enabled.
37.532 + ctx.db.commit()
37.533 + if unload and self.has_pooling:
37.534 + self._unload_context(self._ctx)
37.535 +
37.536 + def rollback():
37.537 + # do db rollback and release the connection if pooling is enabled.
37.538 + ctx.db.rollback()
37.539 + if self.has_pooling:
37.540 + self._unload_context(self._ctx)
37.541 +
37.542 + ctx.commit = commit
37.543 + ctx.rollback = rollback
37.544 +
37.545 + def _unload_context(self, ctx):
37.546 + del ctx.db
37.547 +
37.548 + def _connect(self, keywords):
37.549 + return self.db_module.connect(**keywords)
37.550 +
37.551 + def _connect_with_pooling(self, keywords):
37.552 + def get_pooled_db():
37.553 + from DBUtils import PooledDB
37.554 +
37.555 + # In DBUtils 0.9.3, `dbapi` argument is renamed as `creator`
37.556 + # see Bug#122112
37.557 +
37.558 + if PooledDB.__version__.split('.') < '0.9.3'.split('.'):
37.559 + return PooledDB.PooledDB(dbapi=self.db_module, **keywords)
37.560 + else:
37.561 + return PooledDB.PooledDB(creator=self.db_module, **keywords)
37.562 +
37.563 + if getattr(self, '_pooleddb', None) is None:
37.564 + self._pooleddb = get_pooled_db()
37.565 +
37.566 + return self._pooleddb.connection()
37.567 +
37.568 + def _db_cursor(self):
37.569 + return self.ctx.db.cursor()
37.570 +
37.571 + def _param_marker(self):
37.572 + """Returns parameter marker based on paramstyle attribute if this database."""
37.573 + style = getattr(self, 'paramstyle', 'pyformat')
37.574 +
37.575 + if style == 'qmark':
37.576 + return '?'
37.577 + elif style == 'numeric':
37.578 + return ':1'
37.579 + elif style in ['format', 'pyformat']:
37.580 + return '%s'
37.581 + raise UnknownParamstyle, style
37.582 +
37.583 + def _db_execute(self, cur, sql_query):
37.584 + """executes an sql query"""
37.585 + self.ctx.dbq_count += 1
37.586 +
37.587 + try:
37.588 + a = time.time()
37.589 + query, params = self._process_query(sql_query)
37.590 + out = cur.execute(query, params)
37.591 + b = time.time()
37.592 + except:
37.593 + if self.printing:
37.594 + print >> debug, 'ERR:', str(sql_query)
37.595 + if self.ctx.transactions:
37.596 + self.ctx.transactions[-1].rollback()
37.597 + else:
37.598 + self.ctx.rollback()
37.599 + raise
37.600 +
37.601 + if self.printing:
37.602 + print >> debug, '%s (%s): %s' % (round(b-a, 2), self.ctx.dbq_count, str(sql_query))
37.603 + return out
37.604 +
37.605 + def _process_query(self, sql_query):
37.606 + """Takes the SQLQuery object and returns query string and parameters.
37.607 + """
37.608 + paramstyle = getattr(self, 'paramstyle', 'pyformat')
37.609 + query = sql_query.query(paramstyle)
37.610 + params = sql_query.values()
37.611 + return query, params
37.612 +
37.613 + def _where(self, where, vars):
37.614 + if isinstance(where, (int, long)):
37.615 + where = "id = " + sqlparam(where)
37.616 + #@@@ for backward-compatibility
37.617 + elif isinstance(where, (list, tuple)) and len(where) == 2:
37.618 + where = SQLQuery(where[0], where[1])
37.619 + elif isinstance(where, SQLQuery):
37.620 + pass
37.621 + else:
37.622 + where = reparam(where, vars)
37.623 + return where
37.624 +
37.625 + def query(self, sql_query, vars=None, processed=False, _test=False):
37.626 + """
37.627 + Execute SQL query `sql_query` using dictionary `vars` to interpolate it.
37.628 + If `processed=True`, `vars` is a `reparam`-style list to use
37.629 + instead of interpolating.
37.630 +
37.631 + >>> db = DB(None, {})
37.632 + >>> db.query("SELECT * FROM foo", _test=True)
37.633 + <sql: 'SELECT * FROM foo'>
37.634 + >>> db.query("SELECT * FROM foo WHERE x = $x", vars=dict(x='f'), _test=True)
37.635 + <sql: "SELECT * FROM foo WHERE x = 'f'">
37.636 + >>> db.query("SELECT * FROM foo WHERE x = " + sqlquote('f'), _test=True)
37.637 + <sql: "SELECT * FROM foo WHERE x = 'f'">
37.638 + """
37.639 + if vars is None: vars = {}
37.640 +
37.641 + if not processed and not isinstance(sql_query, SQLQuery):
37.642 + sql_query = reparam(sql_query, vars)
37.643 +
37.644 + if _test: return sql_query
37.645 +
37.646 + db_cursor = self._db_cursor()
37.647 + self._db_execute(db_cursor, sql_query)
37.648 +
37.649 + if db_cursor.description:
37.650 + names = [x[0] for x in db_cursor.description]
37.651 + def iterwrapper():
37.652 + row = db_cursor.fetchone()
37.653 + while row:
37.654 + yield storage(dict(zip(names, row)))
37.655 + row = db_cursor.fetchone()
37.656 + out = iterbetter(iterwrapper())
37.657 + out.__len__ = lambda: int(db_cursor.rowcount)
37.658 + out.list = lambda: [storage(dict(zip(names, x))) \
37.659 + for x in db_cursor.fetchall()]
37.660 + else:
37.661 + out = db_cursor.rowcount
37.662 +
37.663 + if not self.ctx.transactions:
37.664 + self.ctx.commit()
37.665 + return out
37.666 +
37.667 + def select(self, tables, vars=None, what='*', where=None, order=None, group=None,
37.668 + limit=None, offset=None, _test=False):
37.669 + """
37.670 + Selects `what` from `tables` with clauses `where`, `order`,
37.671 + `group`, `limit`, and `offset`. Uses vars to interpolate.
37.672 + Otherwise, each clause can be a SQLQuery.
37.673 +
37.674 + >>> db = DB(None, {})
37.675 + >>> db.select('foo', _test=True)
37.676 + <sql: 'SELECT * FROM foo'>
37.677 + >>> db.select(['foo', 'bar'], where="foo.bar_id = bar.id", limit=5, _test=True)
37.678 + <sql: 'SELECT * FROM foo, bar WHERE foo.bar_id = bar.id LIMIT 5'>
37.679 + """
37.680 + if vars is None: vars = {}
37.681 + sql_clauses = self.sql_clauses(what, tables, where, group, order, limit, offset)
37.682 + clauses = [self.gen_clause(sql, val, vars) for sql, val in sql_clauses if val is not None]
37.683 + qout = SQLQuery.join(clauses)
37.684 + if _test: return qout
37.685 + return self.query(qout, processed=True)
37.686 +
37.687 + def where(self, table, what='*', order=None, group=None, limit=None,
37.688 + offset=None, _test=False, **kwargs):
37.689 + """
37.690 + Selects from `table` where keys are equal to values in `kwargs`.
37.691 +
37.692 + >>> db = DB(None, {})
37.693 + >>> db.where('foo', bar_id=3, _test=True)
37.694 + <sql: 'SELECT * FROM foo WHERE bar_id = 3'>
37.695 + >>> db.where('foo', source=2, crust='dewey', _test=True)
37.696 + <sql: "SELECT * FROM foo WHERE source = 2 AND crust = 'dewey'">
37.697 + >>> db.where('foo', _test=True)
37.698 + <sql: 'SELECT * FROM foo'>
37.699 + """
37.700 + where_clauses = []
37.701 + for k, v in kwargs.iteritems():
37.702 + where_clauses.append(k + ' = ' + sqlquote(v))
37.703 +
37.704 + if where_clauses:
37.705 + where = SQLQuery.join(where_clauses, " AND ")
37.706 + else:
37.707 + where = None
37.708 +
37.709 + return self.select(table, what=what, order=order,
37.710 + group=group, limit=limit, offset=offset, _test=_test,
37.711 + where=where)
37.712 +
37.713 + def sql_clauses(self, what, tables, where, group, order, limit, offset):
37.714 + return (
37.715 + ('SELECT', what),
37.716 + ('FROM', sqllist(tables)),
37.717 + ('WHERE', where),
37.718 + ('GROUP BY', group),
37.719 + ('ORDER BY', order),
37.720 + ('LIMIT', limit),
37.721 + ('OFFSET', offset))
37.722 +
37.723 + def gen_clause(self, sql, val, vars):
37.724 + if isinstance(val, (int, long)):
37.725 + if sql == 'WHERE':
37.726 + nout = 'id = ' + sqlquote(val)
37.727 + else:
37.728 + nout = SQLQuery(val)
37.729 + #@@@
37.730 + elif isinstance(val, (list, tuple)) and len(val) == 2:
37.731 + nout = SQLQuery(val[0], val[1]) # backwards-compatibility
37.732 + elif isinstance(val, SQLQuery):
37.733 + nout = val
37.734 + else:
37.735 + nout = reparam(val, vars)
37.736 +
37.737 + def xjoin(a, b):
37.738 + if a and b: return a + ' ' + b
37.739 + else: return a or b
37.740 +
37.741 + return xjoin(sql, nout)
37.742 +
37.743 + def insert(self, tablename, seqname=None, _test=False, **values):
37.744 + """
37.745 + Inserts `values` into `tablename`. Returns current sequence ID.
37.746 + Set `seqname` to the ID if it's not the default, or to `False`
37.747 + if there isn't one.
37.748 +
37.749 + >>> db = DB(None, {})
37.750 + >>> q = db.insert('foo', name='bob', age=2, created=SQLLiteral('NOW()'), _test=True)
37.751 + >>> q
37.752 + <sql: "INSERT INTO foo (age, name, created) VALUES (2, 'bob', NOW())">
37.753 + >>> q.query()
37.754 + 'INSERT INTO foo (age, name, created) VALUES (%s, %s, NOW())'
37.755 + >>> q.values()
37.756 + [2, 'bob']
37.757 + """
37.758 + def q(x): return "(" + x + ")"
37.759 +
37.760 + if values:
37.761 + _keys = SQLQuery.join(values.keys(), ', ')
37.762 + _values = SQLQuery.join([sqlparam(v) for v in values.values()], ', ')
37.763 + sql_query = "INSERT INTO %s " % tablename + q(_keys) + ' VALUES ' + q(_values)
37.764 + else:
37.765 + sql_query = SQLQuery(self._get_insert_default_values_query(tablename))
37.766 +
37.767 + if _test: return sql_query
37.768 +
37.769 + db_cursor = self._db_cursor()
37.770 + if seqname is not False:
37.771 + sql_query = self._process_insert_query(sql_query, tablename, seqname)
37.772 +
37.773 + if isinstance(sql_query, tuple):
37.774 + # for some databases, a separate query has to be made to find
37.775 + # the id of the inserted row.
37.776 + q1, q2 = sql_query
37.777 + self._db_execute(db_cursor, q1)
37.778 + self._db_execute(db_cursor, q2)
37.779 + else:
37.780 + self._db_execute(db_cursor, sql_query)
37.781 +
37.782 + try:
37.783 + out = db_cursor.fetchone()[0]
37.784 + except Exception:
37.785 + out = None
37.786 +
37.787 + if not self.ctx.transactions:
37.788 + self.ctx.commit()
37.789 + return out
37.790 +
37.791 + def _get_insert_default_values_query(self, table):
37.792 + return "INSERT INTO %s DEFAULT VALUES" % table
37.793 +
37.794 + def multiple_insert(self, tablename, values, seqname=None, _test=False):
37.795 + """
37.796 + Inserts multiple rows into `tablename`. The `values` must be a list of dictioanries,
37.797 + one for each row to be inserted, each with the same set of keys.
37.798 + Returns the list of ids of the inserted rows.
37.799 + Set `seqname` to the ID if it's not the default, or to `False`
37.800 + if there isn't one.
37.801 +
37.802 + >>> db = DB(None, {})
37.803 + >>> db.supports_multiple_insert = True
37.804 + >>> values = [{"name": "foo", "email": "foo@example.com"}, {"name": "bar", "email": "bar@example.com"}]
37.805 + >>> db.multiple_insert('person', values=values, _test=True)
37.806 + <sql: "INSERT INTO person (name, email) VALUES ('foo', 'foo@example.com'), ('bar', 'bar@example.com')">
37.807 + """
37.808 + if not values:
37.809 + return []
37.810 +
37.811 + if not self.supports_multiple_insert:
37.812 + out = [self.insert(tablename, seqname=seqname, _test=_test, **v) for v in values]
37.813 + if seqname is False:
37.814 + return None
37.815 + else:
37.816 + return out
37.817 +
37.818 + keys = values[0].keys()
37.819 + #@@ make sure all keys are valid
37.820 +
37.821 + # make sure all rows have same keys.
37.822 + for v in values:
37.823 + if v.keys() != keys:
37.824 + raise ValueError, 'Bad data'
37.825 +
37.826 + sql_query = SQLQuery('INSERT INTO %s (%s) VALUES ' % (tablename, ', '.join(keys)))
37.827 +
37.828 + for i, row in enumerate(values):
37.829 + if i != 0:
37.830 + sql_query.append(", ")
37.831 + SQLQuery.join([SQLParam(row[k]) for k in keys], sep=", ", target=sql_query, prefix="(", suffix=")")
37.832 +
37.833 + if _test: return sql_query
37.834 +
37.835 + db_cursor = self._db_cursor()
37.836 + if seqname is not False:
37.837 + sql_query = self._process_insert_query(sql_query, tablename, seqname)
37.838 +
37.839 + if isinstance(sql_query, tuple):
37.840 + # for some databases, a separate query has to be made to find
37.841 + # the id of the inserted row.
37.842 + q1, q2 = sql_query
37.843 + self._db_execute(db_cursor, q1)
37.844 + self._db_execute(db_cursor, q2)
37.845 + else:
37.846 + self._db_execute(db_cursor, sql_query)
37.847 +
37.848 + try:
37.849 + out = db_cursor.fetchone()[0]
37.850 + out = range(out-len(values)+1, out+1)
37.851 + except Exception:
37.852 + out = None
37.853 +
37.854 + if not self.ctx.transactions:
37.855 + self.ctx.commit()
37.856 + return out
37.857 +
37.858 +
37.859 + def update(self, tables, where, vars=None, _test=False, **values):
37.860 + """
37.861 + Update `tables` with clause `where` (interpolated using `vars`)
37.862 + and setting `values`.
37.863 +
37.864 + >>> db = DB(None, {})
37.865 + >>> name = 'Joseph'
37.866 + >>> q = db.update('foo', where='name = $name', name='bob', age=2,
37.867 + ... created=SQLLiteral('NOW()'), vars=locals(), _test=True)
37.868 + >>> q
37.869 + <sql: "UPDATE foo SET age = 2, name = 'bob', created = NOW() WHERE name = 'Joseph'">
37.870 + >>> q.query()
37.871 + 'UPDATE foo SET age = %s, name = %s, created = NOW() WHERE name = %s'
37.872 + >>> q.values()
37.873 + [2, 'bob', 'Joseph']
37.874 + """
37.875 + if vars is None: vars = {}
37.876 + where = self._where(where, vars)
37.877 +
37.878 + query = (
37.879 + "UPDATE " + sqllist(tables) +
37.880 + " SET " + sqlwhere(values, ', ') +
37.881 + " WHERE " + where)
37.882 +
37.883 + if _test: return query
37.884 +
37.885 + db_cursor = self._db_cursor()
37.886 + self._db_execute(db_cursor, query)
37.887 + if not self.ctx.transactions:
37.888 + self.ctx.commit()
37.889 + return db_cursor.rowcount
37.890 +
37.891 + def delete(self, table, where, using=None, vars=None, _test=False):
37.892 + """
37.893 + Deletes from `table` with clauses `where` and `using`.
37.894 +
37.895 + >>> db = DB(None, {})
37.896 + >>> name = 'Joe'
37.897 + >>> db.delete('foo', where='name = $name', vars=locals(), _test=True)
37.898 + <sql: "DELETE FROM foo WHERE name = 'Joe'">
37.899 + """
37.900 + if vars is None: vars = {}
37.901 + where = self._where(where, vars)
37.902 +
37.903 + q = 'DELETE FROM ' + table
37.904 + if using: q += ' USING ' + sqllist(using)
37.905 + if where: q += ' WHERE ' + where
37.906 +
37.907 + if _test: return q
37.908 +
37.909 + db_cursor = self._db_cursor()
37.910 + self._db_execute(db_cursor, q)
37.911 + if not self.ctx.transactions:
37.912 + self.ctx.commit()
37.913 + return db_cursor.rowcount
37.914 +
37.915 + def _process_insert_query(self, query, tablename, seqname):
37.916 + return query
37.917 +
37.918 + def transaction(self):
37.919 + """Start a transaction."""
37.920 + return Transaction(self.ctx)
37.921 +
37.922 +class PostgresDB(DB):
37.923 + """Postgres driver."""
37.924 + def __init__(self, **keywords):
37.925 + if 'pw' in keywords:
37.926 + keywords['password'] = keywords.pop('pw')
37.927 +
37.928 + db_module = import_driver(["psycopg2", "psycopg", "pgdb"], preferred=keywords.pop('driver', None))
37.929 + if db_module.__name__ == "psycopg2":
37.930 + import psycopg2.extensions
37.931 + psycopg2.extensions.register_type(psycopg2.extensions.UNICODE)
37.932 +
37.933 + # if db is not provided postgres driver will take it from PGDATABASE environment variable
37.934 + if 'db' in keywords:
37.935 + keywords['database'] = keywords.pop('db')
37.936 +
37.937 + self.dbname = "postgres"
37.938 + self.paramstyle = db_module.paramstyle
37.939 + DB.__init__(self, db_module, keywords)
37.940 + self.supports_multiple_insert = True
37.941 + self._sequences = None
37.942 +
37.943 + def _process_insert_query(self, query, tablename, seqname):
37.944 + if seqname is None:
37.945 + # when seqname is not provided guess the seqname and make sure it exists
37.946 + seqname = tablename + "_id_seq"
37.947 + if seqname not in self._get_all_sequences():
37.948 + seqname = None
37.949 +
37.950 + if seqname:
37.951 + query += "; SELECT currval('%s')" % seqname
37.952 +
37.953 + return query
37.954 +
37.955 + def _get_all_sequences(self):
37.956 + """Query postgres to find names of all sequences used in this database."""
37.957 + if self._sequences is None:
37.958 + q = "SELECT c.relname FROM pg_class c WHERE c.relkind = 'S'"
37.959 + self._sequences = set([c.relname for c in self.query(q)])
37.960 + return self._sequences
37.961 +
37.962 + def _connect(self, keywords):
37.963 + conn = DB._connect(self, keywords)
37.964 + try:
37.965 + conn.set_client_encoding('UTF8')
37.966 + except AttributeError:
37.967 + # fallback for pgdb driver
37.968 + conn.cursor().execute("set client_encoding to 'UTF-8'")
37.969 + return conn
37.970 +
37.971 + def _connect_with_pooling(self, keywords):
37.972 + conn = DB._connect_with_pooling(self, keywords)
37.973 + conn._con._con.set_client_encoding('UTF8')
37.974 + return conn
37.975 +
37.976 +class MySQLDB(DB):
37.977 + def __init__(self, **keywords):
37.978 + import MySQLdb as db
37.979 + if 'pw' in keywords:
37.980 + keywords['passwd'] = keywords['pw']
37.981 + del keywords['pw']
37.982 +
37.983 + if 'charset' not in keywords:
37.984 + keywords['charset'] = 'utf8'
37.985 + elif keywords['charset'] is None:
37.986 + del keywords['charset']
37.987 +
37.988 + self.paramstyle = db.paramstyle = 'pyformat' # it's both, like psycopg
37.989 + self.dbname = "mysql"
37.990 + DB.__init__(self, db, keywords)
37.991 + self.supports_multiple_insert = True
37.992 +
37.993 + def _process_insert_query(self, query, tablename, seqname):
37.994 + return query, SQLQuery('SELECT last_insert_id();')
37.995 +
37.996 + def _get_insert_default_values_query(self, table):
37.997 + return "INSERT INTO %s () VALUES()" % table
37.998 +
37.999 +def import_driver(drivers, preferred=None):
37.1000 + """Import the first available driver or preferred driver.
37.1001 + """
37.1002 + if preferred:
37.1003 + drivers = [preferred]
37.1004 +
37.1005 + for d in drivers:
37.1006 + try:
37.1007 + return __import__(d, None, None, ['x'])
37.1008 + except ImportError:
37.1009 + pass
37.1010 + raise ImportError("Unable to import " + " or ".join(drivers))
37.1011 +
37.1012 +class SqliteDB(DB):
37.1013 + def __init__(self, **keywords):
37.1014 + db = import_driver(["sqlite3", "pysqlite2.dbapi2", "sqlite"], preferred=keywords.pop('driver', None))
37.1015 +
37.1016 + if db.__name__ in ["sqlite3", "pysqlite2.dbapi2"]:
37.1017 + db.paramstyle = 'qmark'
37.1018 +
37.1019 + # sqlite driver doesn't create datatime objects for timestamp columns unless `detect_types` option is passed.
37.1020 + # It seems to be supported in sqlite3 and pysqlite2 drivers, not surte about sqlite.
37.1021 + keywords.setdefault('detect_types', db.PARSE_DECLTYPES)
37.1022 +
37.1023 + self.paramstyle = db.paramstyle
37.1024 + keywords['database'] = keywords.pop('db')
37.1025 + keywords['pooling'] = False # sqlite don't allows connections to be shared by threads
37.1026 + self.dbname = "sqlite"
37.1027 + DB.__init__(self, db, keywords)
37.1028 +
37.1029 + def _process_insert_query(self, query, tablename, seqname):
37.1030 + return query, SQLQuery('SELECT last_insert_rowid();')
37.1031 +
37.1032 + def query(self, *a, **kw):
37.1033 + out = DB.query(self, *a, **kw)
37.1034 + if isinstance(out, iterbetter):
37.1035 + del out.__len__
37.1036 + return out
37.1037 +
37.1038 +class FirebirdDB(DB):
37.1039 + """Firebird Database.
37.1040 + """
37.1041 + def __init__(self, **keywords):
37.1042 + try:
37.1043 + import kinterbasdb as db
37.1044 + except Exception:
37.1045 + db = None
37.1046 + pass
37.1047 + if 'pw' in keywords:
37.1048 + keywords['passwd'] = keywords['pw']
37.1049 + del keywords['pw']
37.1050 + keywords['database'] = keywords['db']
37.1051 + del keywords['db']
37.1052 + DB.__init__(self, db, keywords)
37.1053 +
37.1054 + def delete(self, table, where=None, using=None, vars=None, _test=False):
37.1055 + # firebird doesn't support using clause
37.1056 + using=None
37.1057 + return DB.delete(self, table, where, using, vars, _test)
37.1058 +
37.1059 + def sql_clauses(self, what, tables, where, group, order, limit, offset):
37.1060 + return (
37.1061 + ('SELECT', ''),
37.1062 + ('FIRST', limit),
37.1063 + ('SKIP', offset),
37.1064 + ('', what),
37.1065 + ('FROM', sqllist(tables)),
37.1066 + ('WHERE', where),
37.1067 + ('GROUP BY', group),
37.1068 + ('ORDER BY', order)
37.1069 + )
37.1070 +
37.1071 +class MSSQLDB(DB):
37.1072 + def __init__(self, **keywords):
37.1073 + import pymssql as db
37.1074 + if 'pw' in keywords:
37.1075 + keywords['password'] = keywords.pop('pw')
37.1076 + keywords['database'] = keywords.pop('db')
37.1077 + self.dbname = "mssql"
37.1078 + DB.__init__(self, db, keywords)
37.1079 +
37.1080 + def _process_query(self, sql_query):
37.1081 + """Takes the SQLQuery object and returns query string and parameters.
37.1082 + """
37.1083 + # MSSQLDB expects params to be a tuple.
37.1084 + # Overwriting the default implementation to convert params to tuple.
37.1085 + paramstyle = getattr(self, 'paramstyle', 'pyformat')
37.1086 + query = sql_query.query(paramstyle)
37.1087 + params = sql_query.values()
37.1088 + return query, tuple(params)
37.1089 +
37.1090 + def sql_clauses(self, what, tables, where, group, order, limit, offset):
37.1091 + return (
37.1092 + ('SELECT', what),
37.1093 + ('TOP', limit),
37.1094 + ('FROM', sqllist(tables)),
37.1095 + ('WHERE', where),
37.1096 + ('GROUP BY', group),
37.1097 + ('ORDER BY', order),
37.1098 + ('OFFSET', offset))
37.1099 +
37.1100 + def _test(self):
37.1101 + """Test LIMIT.
37.1102 +
37.1103 + Fake presence of pymssql module for running tests.
37.1104 + >>> import sys
37.1105 + >>> sys.modules['pymssql'] = sys.modules['sys']
37.1106 +
37.1107 + MSSQL has TOP clause instead of LIMIT clause.
37.1108 + >>> db = MSSQLDB(db='test', user='joe', pw='secret')
37.1109 + >>> db.select('foo', limit=4, _test=True)
37.1110 + <sql: 'SELECT * TOP 4 FROM foo'>
37.1111 + """
37.1112 + pass
37.1113 +
37.1114 +class OracleDB(DB):
37.1115 + def __init__(self, **keywords):
37.1116 + import cx_Oracle as db
37.1117 + if 'pw' in keywords:
37.1118 + keywords['password'] = keywords.pop('pw')
37.1119 +
37.1120 + #@@ TODO: use db.makedsn if host, port is specified
37.1121 + keywords['dsn'] = keywords.pop('db')
37.1122 + self.dbname = 'oracle'
37.1123 + db.paramstyle = 'numeric'
37.1124 + self.paramstyle = db.paramstyle
37.1125 +
37.1126 + # oracle doesn't support pooling
37.1127 + keywords.pop('pooling', None)
37.1128 + DB.__init__(self, db, keywords)
37.1129 +
37.1130 + def _process_insert_query(self, query, tablename, seqname):
37.1131 + if seqname is None:
37.1132 + # It is not possible to get seq name from table name in Oracle
37.1133 + return query
37.1134 + else:
37.1135 + return query + "; SELECT %s.currval FROM dual" % seqname
37.1136 +
37.1137 +_databases = {}
37.1138 +def database(dburl=None, **params):
37.1139 + """Creates appropriate database using params.
37.1140 +
37.1141 + Pooling will be enabled if DBUtils module is available.
37.1142 + Pooling can be disabled by passing pooling=False in params.
37.1143 + """
37.1144 + dbn = params.pop('dbn')
37.1145 + if dbn in _databases:
37.1146 + return _databases[dbn](**params)
37.1147 + else:
37.1148 + raise UnknownDB, dbn
37.1149 +
37.1150 +def register_database(name, clazz):
37.1151 + """
37.1152 + Register a database.
37.1153 +
37.1154 + >>> class LegacyDB(DB):
37.1155 + ... def __init__(self, **params):
37.1156 + ... pass
37.1157 + ...
37.1158 + >>> register_database('legacy', LegacyDB)
37.1159 + >>> db = database(dbn='legacy', db='test', user='joe', passwd='secret')
37.1160 + """
37.1161 + _databases[name] = clazz
37.1162 +
37.1163 +register_database('mysql', MySQLDB)
37.1164 +register_database('postgres', PostgresDB)
37.1165 +register_database('sqlite', SqliteDB)
37.1166 +register_database('firebird', FirebirdDB)
37.1167 +register_database('mssql', MSSQLDB)
37.1168 +register_database('oracle', OracleDB)
37.1169 +
37.1170 +def _interpolate(format):
37.1171 + """
37.1172 + Takes a format string and returns a list of 2-tuples of the form
37.1173 + (boolean, string) where boolean says whether string should be evaled
37.1174 + or not.
37.1175 +
37.1176 + from <http://lfw.org/python/Itpl.py> (public domain, Ka-Ping Yee)
37.1177 + """
37.1178 + from tokenize import tokenprog
37.1179 +
37.1180 + def matchorfail(text, pos):
37.1181 + match = tokenprog.match(text, pos)
37.1182 + if match is None:
37.1183 + raise _ItplError(text, pos)
37.1184 + return match, match.end()
37.1185 +
37.1186 + namechars = "abcdefghijklmnopqrstuvwxyz" \
37.1187 + "ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789_";
37.1188 + chunks = []
37.1189 + pos = 0
37.1190 +
37.1191 + while 1:
37.1192 + dollar = format.find("$", pos)
37.1193 + if dollar < 0:
37.1194 + break
37.1195 + nextchar = format[dollar + 1]
37.1196 +
37.1197 + if nextchar == "{":
37.1198 + chunks.append((0, format[pos:dollar]))
37.1199 + pos, level = dollar + 2, 1
37.1200 + while level:
37.1201 + match, pos = matchorfail(format, pos)
37.1202 + tstart, tend = match.regs[3]
37.1203 + token = format[tstart:tend]
37.1204 + if token == "{":
37.1205 + level = level + 1
37.1206 + elif token == "}":
37.1207 + level = level - 1
37.1208 + chunks.append((1, format[dollar + 2:pos - 1]))
37.1209 +
37.1210 + elif nextchar in namechars:
37.1211 + chunks.append((0, format[pos:dollar]))
37.1212 + match, pos = matchorfail(format, dollar + 1)
37.1213 + while pos < len(format):
37.1214 + if format[pos] == "." and \
37.1215 + pos + 1 < len(format) and format[pos + 1] in namechars:
37.1216 + match, pos = matchorfail(format, pos + 1)
37.1217 + elif format[pos] in "([":
37.1218 + pos, level = pos + 1, 1
37.1219 + while level:
37.1220 + match, pos = matchorfail(format, pos)
37.1221 + tstart, tend = match.regs[3]
37.1222 + token = format[tstart:tend]
37.1223 + if token[0] in "([":
37.1224 + level = level + 1
37.1225 + elif token[0] in ")]":
37.1226 + level = level - 1
37.1227 + else:
37.1228 + break
37.1229 + chunks.append((1, format[dollar + 1:pos]))
37.1230 + else:
37.1231 + chunks.append((0, format[pos:dollar + 1]))
37.1232 + pos = dollar + 1 + (nextchar == "$")
37.1233 +
37.1234 + if pos < len(format):
37.1235 + chunks.append((0, format[pos:]))
37.1236 + return chunks
37.1237 +
37.1238 +if __name__ == "__main__":
37.1239 + import doctest
37.1240 + doctest.testmod()
38.1 --- /dev/null Thu Jan 01 00:00:00 1970 +0000
38.2 +++ b/OpenSecurity/install/web.py-0.37/build/lib/web/debugerror.py Mon Dec 02 14:02:05 2013 +0100
38.3 @@ -0,0 +1,354 @@
38.4 +"""
38.5 +pretty debug errors
38.6 +(part of web.py)
38.7 +
38.8 +portions adapted from Django <djangoproject.com>
38.9 +Copyright (c) 2005, the Lawrence Journal-World
38.10 +Used under the modified BSD license:
38.11 +http://www.xfree86.org/3.3.6/COPYRIGHT2.html#5
38.12 +"""
38.13 +
38.14 +__all__ = ["debugerror", "djangoerror", "emailerrors"]
38.15 +
38.16 +import sys, urlparse, pprint, traceback
38.17 +from template import Template
38.18 +from net import websafe
38.19 +from utils import sendmail, safestr
38.20 +import webapi as web
38.21 +
38.22 +import os, os.path
38.23 +whereami = os.path.join(os.getcwd(), __file__)
38.24 +whereami = os.path.sep.join(whereami.split(os.path.sep)[:-1])
38.25 +djangoerror_t = """\
38.26 +$def with (exception_type, exception_value, frames)
38.27 +<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
38.28 +<html lang="en">
38.29 +<head>
38.30 + <meta http-equiv="content-type" content="text/html; charset=utf-8" />
38.31 + <meta name="robots" content="NONE,NOARCHIVE" />
38.32 + <title>$exception_type at $ctx.path</title>
38.33 + <style type="text/css">
38.34 + html * { padding:0; margin:0; }
38.35 + body * { padding:10px 20px; }
38.36 + body * * { padding:0; }
38.37 + body { font:small sans-serif; }
38.38 + body>div { border-bottom:1px solid #ddd; }
38.39 + h1 { font-weight:normal; }
38.40 + h2 { margin-bottom:.8em; }
38.41 + h2 span { font-size:80%; color:#666; font-weight:normal; }
38.42 + h3 { margin:1em 0 .5em 0; }
38.43 + h4 { margin:0 0 .5em 0; font-weight: normal; }
38.44 + table {
38.45 + border:1px solid #ccc; border-collapse: collapse; background:white; }
38.46 + tbody td, tbody th { vertical-align:top; padding:2px 3px; }
38.47 + thead th {
38.48 + padding:1px 6px 1px 3px; background:#fefefe; text-align:left;
38.49 + font-weight:normal; font-size:11px; border:1px solid #ddd; }
38.50 + tbody th { text-align:right; color:#666; padding-right:.5em; }
38.51 + table.vars { margin:5px 0 2px 40px; }
38.52 + table.vars td, table.req td { font-family:monospace; }
38.53 + table td.code { width:100%;}
38.54 + table td.code div { overflow:hidden; }
38.55 + table.source th { color:#666; }
38.56 + table.source td {
38.57 + font-family:monospace; white-space:pre; border-bottom:1px solid #eee; }
38.58 + ul.traceback { list-style-type:none; }
38.59 + ul.traceback li.frame { margin-bottom:1em; }
38.60 + div.context { margin: 10px 0; }
38.61 + div.context ol {
38.62 + padding-left:30px; margin:0 10px; list-style-position: inside; }
38.63 + div.context ol li {
38.64 + font-family:monospace; white-space:pre; color:#666; cursor:pointer; }
38.65 + div.context ol.context-line li { color:black; background-color:#ccc; }
38.66 + div.context ol.context-line li span { float: right; }
38.67 + div.commands { margin-left: 40px; }
38.68 + div.commands a { color:black; text-decoration:none; }
38.69 + #summary { background: #ffc; }
38.70 + #summary h2 { font-weight: normal; color: #666; }
38.71 + #explanation { background:#eee; }
38.72 + #template, #template-not-exist { background:#f6f6f6; }
38.73 + #template-not-exist ul { margin: 0 0 0 20px; }
38.74 + #traceback { background:#eee; }
38.75 + #requestinfo { background:#f6f6f6; padding-left:120px; }
38.76 + #summary table { border:none; background:transparent; }
38.77 + #requestinfo h2, #requestinfo h3 { position:relative; margin-left:-100px; }
38.78 + #requestinfo h3 { margin-bottom:-1em; }
38.79 + .error { background: #ffc; }
38.80 + .specific { color:#cc3300; font-weight:bold; }
38.81 + </style>
38.82 + <script type="text/javascript">
38.83 + //<!--
38.84 + function getElementsByClassName(oElm, strTagName, strClassName){
38.85 + // Written by Jonathan Snook, http://www.snook.ca/jon;
38.86 + // Add-ons by Robert Nyman, http://www.robertnyman.com
38.87 + var arrElements = (strTagName == "*" && document.all)? document.all :
38.88 + oElm.getElementsByTagName(strTagName);
38.89 + var arrReturnElements = new Array();
38.90 + strClassName = strClassName.replace(/\-/g, "\\-");
38.91 + var oRegExp = new RegExp("(^|\\s)" + strClassName + "(\\s|$$)");
38.92 + var oElement;
38.93 + for(var i=0; i<arrElements.length; i++){
38.94 + oElement = arrElements[i];
38.95 + if(oRegExp.test(oElement.className)){
38.96 + arrReturnElements.push(oElement);
38.97 + }
38.98 + }
38.99 + return (arrReturnElements)
38.100 + }
38.101 + function hideAll(elems) {
38.102 + for (var e = 0; e < elems.length; e++) {
38.103 + elems[e].style.display = 'none';
38.104 + }
38.105 + }
38.106 + window.onload = function() {
38.107 + hideAll(getElementsByClassName(document, 'table', 'vars'));
38.108 + hideAll(getElementsByClassName(document, 'ol', 'pre-context'));
38.109 + hideAll(getElementsByClassName(document, 'ol', 'post-context'));
38.110 + }
38.111 + function toggle() {
38.112 + for (var i = 0; i < arguments.length; i++) {
38.113 + var e = document.getElementById(arguments[i]);
38.114 + if (e) {
38.115 + e.style.display = e.style.display == 'none' ? 'block' : 'none';
38.116 + }
38.117 + }
38.118 + return false;
38.119 + }
38.120 + function varToggle(link, id) {
38.121 + toggle('v' + id);
38.122 + var s = link.getElementsByTagName('span')[0];
38.123 + var uarr = String.fromCharCode(0x25b6);
38.124 + var darr = String.fromCharCode(0x25bc);
38.125 + s.innerHTML = s.innerHTML == uarr ? darr : uarr;
38.126 + return false;
38.127 + }
38.128 + //-->
38.129 + </script>
38.130 +</head>
38.131 +<body>
38.132 +
38.133 +$def dicttable (d, kls='req', id=None):
38.134 + $ items = d and d.items() or []
38.135 + $items.sort()
38.136 + $:dicttable_items(items, kls, id)
38.137 +
38.138 +$def dicttable_items(items, kls='req', id=None):
38.139 + $if items:
38.140 + <table class="$kls"
38.141 + $if id: id="$id"
38.142 + ><thead><tr><th>Variable</th><th>Value</th></tr></thead>
38.143 + <tbody>
38.144 + $for k, v in items:
38.145 + <tr><td>$k</td><td class="code"><div>$prettify(v)</div></td></tr>
38.146 + </tbody>
38.147 + </table>
38.148 + $else:
38.149 + <p>No data.</p>
38.150 +
38.151 +<div id="summary">
38.152 + <h1>$exception_type at $ctx.path</h1>
38.153 + <h2>$exception_value</h2>
38.154 + <table><tr>
38.155 + <th>Python</th>
38.156 + <td>$frames[0].filename in $frames[0].function, line $frames[0].lineno</td>
38.157 + </tr><tr>
38.158 + <th>Web</th>
38.159 + <td>$ctx.method $ctx.home$ctx.path</td>
38.160 + </tr></table>
38.161 +</div>
38.162 +<div id="traceback">
38.163 +<h2>Traceback <span>(innermost first)</span></h2>
38.164 +<ul class="traceback">
38.165 +$for frame in frames:
38.166 + <li class="frame">
38.167 + <code>$frame.filename</code> in <code>$frame.function</code>
38.168 + $if frame.context_line is not None:
38.169 + <div class="context" id="c$frame.id">
38.170 + $if frame.pre_context:
38.171 + <ol start="$frame.pre_context_lineno" class="pre-context" id="pre$frame.id">
38.172 + $for line in frame.pre_context:
38.173 + <li onclick="toggle('pre$frame.id', 'post$frame.id')">$line</li>
38.174 + </ol>
38.175 + <ol start="$frame.lineno" class="context-line"><li onclick="toggle('pre$frame.id', 'post$frame.id')">$frame.context_line <span>...</span></li></ol>
38.176 + $if frame.post_context:
38.177 + <ol start='${frame.lineno + 1}' class="post-context" id="post$frame.id">
38.178 + $for line in frame.post_context:
38.179 + <li onclick="toggle('pre$frame.id', 'post$frame.id')">$line</li>
38.180 + </ol>
38.181 + </div>
38.182 +
38.183 + $if frame.vars:
38.184 + <div class="commands">
38.185 + <a href='#' onclick="return varToggle(this, '$frame.id')"><span>▶</span> Local vars</a>
38.186 + $# $inspect.formatargvalues(*inspect.getargvalues(frame['tb'].tb_frame))
38.187 + </div>
38.188 + $:dicttable(frame.vars, kls='vars', id=('v' + str(frame.id)))
38.189 + </li>
38.190 + </ul>
38.191 +</div>
38.192 +
38.193 +<div id="requestinfo">
38.194 +$if ctx.output or ctx.headers:
38.195 + <h2>Response so far</h2>
38.196 + <h3>HEADERS</h3>
38.197 + $:dicttable_items(ctx.headers)
38.198 +
38.199 + <h3>BODY</h3>
38.200 + <p class="req" style="padding-bottom: 2em"><code>
38.201 + $ctx.output
38.202 + </code></p>
38.203 +
38.204 +<h2>Request information</h2>
38.205 +
38.206 +<h3>INPUT</h3>
38.207 +$:dicttable(web.input(_unicode=False))
38.208 +
38.209 +<h3 id="cookie-info">COOKIES</h3>
38.210 +$:dicttable(web.cookies())
38.211 +
38.212 +<h3 id="meta-info">META</h3>
38.213 +$ newctx = [(k, v) for (k, v) in ctx.iteritems() if not k.startswith('_') and not isinstance(v, dict)]
38.214 +$:dicttable(dict(newctx))
38.215 +
38.216 +<h3 id="meta-info">ENVIRONMENT</h3>
38.217 +$:dicttable(ctx.env)
38.218 +</div>
38.219 +
38.220 +<div id="explanation">
38.221 + <p>
38.222 + You're seeing this error because you have <code>web.config.debug</code>
38.223 + set to <code>True</code>. Set that to <code>False</code> if you don't want to see this.
38.224 + </p>
38.225 +</div>
38.226 +
38.227 +</body>
38.228 +</html>
38.229 +"""
38.230 +
38.231 +djangoerror_r = None
38.232 +
38.233 +def djangoerror():
38.234 + def _get_lines_from_file(filename, lineno, context_lines):
38.235 + """
38.236 + Returns context_lines before and after lineno from file.
38.237 + Returns (pre_context_lineno, pre_context, context_line, post_context).
38.238 + """
38.239 + try:
38.240 + source = open(filename).readlines()
38.241 + lower_bound = max(0, lineno - context_lines)
38.242 + upper_bound = lineno + context_lines
38.243 +
38.244 + pre_context = \
38.245 + [line.strip('\n') for line in source[lower_bound:lineno]]
38.246 + context_line = source[lineno].strip('\n')
38.247 + post_context = \
38.248 + [line.strip('\n') for line in source[lineno + 1:upper_bound]]
38.249 +
38.250 + return lower_bound, pre_context, context_line, post_context
38.251 + except (OSError, IOError, IndexError):
38.252 + return None, [], None, []
38.253 +
38.254 + exception_type, exception_value, tback = sys.exc_info()
38.255 + frames = []
38.256 + while tback is not None:
38.257 + filename = tback.tb_frame.f_code.co_filename
38.258 + function = tback.tb_frame.f_code.co_name
38.259 + lineno = tback.tb_lineno - 1
38.260 +
38.261 + # hack to get correct line number for templates
38.262 + lineno += tback.tb_frame.f_locals.get("__lineoffset__", 0)
38.263 +
38.264 + pre_context_lineno, pre_context, context_line, post_context = \
38.265 + _get_lines_from_file(filename, lineno, 7)
38.266 +
38.267 + if '__hidetraceback__' not in tback.tb_frame.f_locals:
38.268 + frames.append(web.storage({
38.269 + 'tback': tback,
38.270 + 'filename': filename,
38.271 + 'function': function,
38.272 + 'lineno': lineno,
38.273 + 'vars': tback.tb_frame.f_locals,
38.274 + 'id': id(tback),
38.275 + 'pre_context': pre_context,
38.276 + 'context_line': context_line,
38.277 + 'post_context': post_context,
38.278 + 'pre_context_lineno': pre_context_lineno,
38.279 + }))
38.280 + tback = tback.tb_next
38.281 + frames.reverse()
38.282 + urljoin = urlparse.urljoin
38.283 + def prettify(x):
38.284 + try:
38.285 + out = pprint.pformat(x)
38.286 + except Exception, e:
38.287 + out = '[could not display: <' + e.__class__.__name__ + \
38.288 + ': '+str(e)+'>]'
38.289 + return out
38.290 +
38.291 + global djangoerror_r
38.292 + if djangoerror_r is None:
38.293 + djangoerror_r = Template(djangoerror_t, filename=__file__, filter=websafe)
38.294 +
38.295 + t = djangoerror_r
38.296 + globals = {'ctx': web.ctx, 'web':web, 'dict':dict, 'str':str, 'prettify': prettify}
38.297 + t.t.func_globals.update(globals)
38.298 + return t(exception_type, exception_value, frames)
38.299 +
38.300 +def debugerror():
38.301 + """
38.302 + A replacement for `internalerror` that presents a nice page with lots
38.303 + of debug information for the programmer.
38.304 +
38.305 + (Based on the beautiful 500 page from [Django](http://djangoproject.com/),
38.306 + designed by [Wilson Miner](http://wilsonminer.com/).)
38.307 + """
38.308 + return web._InternalError(djangoerror())
38.309 +
38.310 +def emailerrors(to_address, olderror, from_address=None):
38.311 + """
38.312 + Wraps the old `internalerror` handler (pass as `olderror`) to
38.313 + additionally email all errors to `to_address`, to aid in
38.314 + debugging production websites.
38.315 +
38.316 + Emails contain a normal text traceback as well as an
38.317 + attachment containing the nice `debugerror` page.
38.318 + """
38.319 + from_address = from_address or to_address
38.320 +
38.321 + def emailerrors_internal():
38.322 + error = olderror()
38.323 + tb = sys.exc_info()
38.324 + error_name = tb[0]
38.325 + error_value = tb[1]
38.326 + tb_txt = ''.join(traceback.format_exception(*tb))
38.327 + path = web.ctx.path
38.328 + request = web.ctx.method + ' ' + web.ctx.home + web.ctx.fullpath
38.329 +
38.330 + message = "\n%s\n\n%s\n\n" % (request, tb_txt)
38.331 +
38.332 + sendmail(
38.333 + "your buggy site <%s>" % from_address,
38.334 + "the bugfixer <%s>" % to_address,
38.335 + "bug: %(error_name)s: %(error_value)s (%(path)s)" % locals(),
38.336 + message,
38.337 + attachments=[
38.338 + dict(filename="bug.html", content=safestr(djangoerror()))
38.339 + ],
38.340 + )
38.341 + return error
38.342 +
38.343 + return emailerrors_internal
38.344 +
38.345 +if __name__ == "__main__":
38.346 + urls = (
38.347 + '/', 'index'
38.348 + )
38.349 + from application import application
38.350 + app = application(urls, globals())
38.351 + app.internalerror = debugerror
38.352 +
38.353 + class index:
38.354 + def GET(self):
38.355 + thisdoesnotexist
38.356 +
38.357 + app.run()
39.1 --- /dev/null Thu Jan 01 00:00:00 1970 +0000
39.2 +++ b/OpenSecurity/install/web.py-0.37/build/lib/web/form.py Mon Dec 02 14:02:05 2013 +0100
39.3 @@ -0,0 +1,410 @@
39.4 +"""
39.5 +HTML forms
39.6 +(part of web.py)
39.7 +"""
39.8 +
39.9 +import copy, re
39.10 +import webapi as web
39.11 +import utils, net
39.12 +
39.13 +def attrget(obj, attr, value=None):
39.14 + try:
39.15 + if hasattr(obj, 'has_key') and obj.has_key(attr):
39.16 + return obj[attr]
39.17 + except TypeError:
39.18 + # Handle the case where has_key takes different number of arguments.
39.19 + # This is the case with Model objects on appengine. See #134
39.20 + pass
39.21 + if hasattr(obj, attr):
39.22 + return getattr(obj, attr)
39.23 + return value
39.24 +
39.25 +class Form(object):
39.26 + r"""
39.27 + HTML form.
39.28 +
39.29 + >>> f = Form(Textbox("x"))
39.30 + >>> f.render()
39.31 + u'<table>\n <tr><th><label for="x">x</label></th><td><input type="text" id="x" name="x"/></td></tr>\n</table>'
39.32 + """
39.33 + def __init__(self, *inputs, **kw):
39.34 + self.inputs = inputs
39.35 + self.valid = True
39.36 + self.note = None
39.37 + self.validators = kw.pop('validators', [])
39.38 +
39.39 + def __call__(self, x=None):
39.40 + o = copy.deepcopy(self)
39.41 + if x: o.validates(x)
39.42 + return o
39.43 +
39.44 + def render(self):
39.45 + out = ''
39.46 + out += self.rendernote(self.note)
39.47 + out += '<table>\n'
39.48 +
39.49 + for i in self.inputs:
39.50 + html = utils.safeunicode(i.pre) + i.render() + self.rendernote(i.note) + utils.safeunicode(i.post)
39.51 + if i.is_hidden():
39.52 + out += ' <tr style="display: none;"><th></th><td>%s</td></tr>\n' % (html)
39.53 + else:
39.54 + out += ' <tr><th><label for="%s">%s</label></th><td>%s</td></tr>\n' % (i.id, net.websafe(i.description), html)
39.55 + out += "</table>"
39.56 + return out
39.57 +
39.58 + def render_css(self):
39.59 + out = []
39.60 + out.append(self.rendernote(self.note))
39.61 + for i in self.inputs:
39.62 + if not i.is_hidden():
39.63 + out.append('<label for="%s">%s</label>' % (i.id, net.websafe(i.description)))
39.64 + out.append(i.pre)
39.65 + out.append(i.render())
39.66 + out.append(self.rendernote(i.note))
39.67 + out.append(i.post)
39.68 + out.append('\n')
39.69 + return ''.join(out)
39.70 +
39.71 + def rendernote(self, note):
39.72 + if note: return '<strong class="wrong">%s</strong>' % net.websafe(note)
39.73 + else: return ""
39.74 +
39.75 + def validates(self, source=None, _validate=True, **kw):
39.76 + source = source or kw or web.input()
39.77 + out = True
39.78 + for i in self.inputs:
39.79 + v = attrget(source, i.name)
39.80 + if _validate:
39.81 + out = i.validate(v) and out
39.82 + else:
39.83 + i.set_value(v)
39.84 + if _validate:
39.85 + out = out and self._validate(source)
39.86 + self.valid = out
39.87 + return out
39.88 +
39.89 + def _validate(self, value):
39.90 + self.value = value
39.91 + for v in self.validators:
39.92 + if not v.valid(value):
39.93 + self.note = v.msg
39.94 + return False
39.95 + return True
39.96 +
39.97 + def fill(self, source=None, **kw):
39.98 + return self.validates(source, _validate=False, **kw)
39.99 +
39.100 + def __getitem__(self, i):
39.101 + for x in self.inputs:
39.102 + if x.name == i: return x
39.103 + raise KeyError, i
39.104 +
39.105 + def __getattr__(self, name):
39.106 + # don't interfere with deepcopy
39.107 + inputs = self.__dict__.get('inputs') or []
39.108 + for x in inputs:
39.109 + if x.name == name: return x
39.110 + raise AttributeError, name
39.111 +
39.112 + def get(self, i, default=None):
39.113 + try:
39.114 + return self[i]
39.115 + except KeyError:
39.116 + return default
39.117 +
39.118 + def _get_d(self): #@@ should really be form.attr, no?
39.119 + return utils.storage([(i.name, i.get_value()) for i in self.inputs])
39.120 + d = property(_get_d)
39.121 +
39.122 +class Input(object):
39.123 + def __init__(self, name, *validators, **attrs):
39.124 + self.name = name
39.125 + self.validators = validators
39.126 + self.attrs = attrs = AttributeList(attrs)
39.127 +
39.128 + self.description = attrs.pop('description', name)
39.129 + self.value = attrs.pop('value', None)
39.130 + self.pre = attrs.pop('pre', "")
39.131 + self.post = attrs.pop('post', "")
39.132 + self.note = None
39.133 +
39.134 + self.id = attrs.setdefault('id', self.get_default_id())
39.135 +
39.136 + if 'class_' in attrs:
39.137 + attrs['class'] = attrs['class_']
39.138 + del attrs['class_']
39.139 +
39.140 + def is_hidden(self):
39.141 + return False
39.142 +
39.143 + def get_type(self):
39.144 + raise NotImplementedError
39.145 +
39.146 + def get_default_id(self):
39.147 + return self.name
39.148 +
39.149 + def validate(self, value):
39.150 + self.set_value(value)
39.151 +
39.152 + for v in self.validators:
39.153 + if not v.valid(value):
39.154 + self.note = v.msg
39.155 + return False
39.156 + return True
39.157 +
39.158 + def set_value(self, value):
39.159 + self.value = value
39.160 +
39.161 + def get_value(self):
39.162 + return self.value
39.163 +
39.164 + def render(self):
39.165 + attrs = self.attrs.copy()
39.166 + attrs['type'] = self.get_type()
39.167 + if self.value is not None:
39.168 + attrs['value'] = self.value
39.169 + attrs['name'] = self.name
39.170 + return '<input %s/>' % attrs
39.171 +
39.172 + def rendernote(self, note):
39.173 + if note: return '<strong class="wrong">%s</strong>' % net.websafe(note)
39.174 + else: return ""
39.175 +
39.176 + def addatts(self):
39.177 + # add leading space for backward-compatibility
39.178 + return " " + str(self.attrs)
39.179 +
39.180 +class AttributeList(dict):
39.181 + """List of atributes of input.
39.182 +
39.183 + >>> a = AttributeList(type='text', name='x', value=20)
39.184 + >>> a
39.185 + <attrs: 'type="text" name="x" value="20"'>
39.186 + """
39.187 + def copy(self):
39.188 + return AttributeList(self)
39.189 +
39.190 + def __str__(self):
39.191 + return " ".join(['%s="%s"' % (k, net.websafe(v)) for k, v in self.items()])
39.192 +
39.193 + def __repr__(self):
39.194 + return '<attrs: %s>' % repr(str(self))
39.195 +
39.196 +class Textbox(Input):
39.197 + """Textbox input.
39.198 +
39.199 + >>> Textbox(name='foo', value='bar').render()
39.200 + u'<input type="text" id="foo" value="bar" name="foo"/>'
39.201 + >>> Textbox(name='foo', value=0).render()
39.202 + u'<input type="text" id="foo" value="0" name="foo"/>'
39.203 + """
39.204 + def get_type(self):
39.205 + return 'text'
39.206 +
39.207 +class Password(Input):
39.208 + """Password input.
39.209 +
39.210 + >>> Password(name='password', value='secret').render()
39.211 + u'<input type="password" id="password" value="secret" name="password"/>'
39.212 + """
39.213 +
39.214 + def get_type(self):
39.215 + return 'password'
39.216 +
39.217 +class Textarea(Input):
39.218 + """Textarea input.
39.219 +
39.220 + >>> Textarea(name='foo', value='bar').render()
39.221 + u'<textarea id="foo" name="foo">bar</textarea>'
39.222 + """
39.223 + def render(self):
39.224 + attrs = self.attrs.copy()
39.225 + attrs['name'] = self.name
39.226 + value = net.websafe(self.value or '')
39.227 + return '<textarea %s>%s</textarea>' % (attrs, value)
39.228 +
39.229 +class Dropdown(Input):
39.230 + r"""Dropdown/select input.
39.231 +
39.232 + >>> Dropdown(name='foo', args=['a', 'b', 'c'], value='b').render()
39.233 + u'<select id="foo" name="foo">\n <option value="a">a</option>\n <option selected="selected" value="b">b</option>\n <option value="c">c</option>\n</select>\n'
39.234 + >>> Dropdown(name='foo', args=[('a', 'aa'), ('b', 'bb'), ('c', 'cc')], value='b').render()
39.235 + u'<select id="foo" name="foo">\n <option value="a">aa</option>\n <option selected="selected" value="b">bb</option>\n <option value="c">cc</option>\n</select>\n'
39.236 + """
39.237 + def __init__(self, name, args, *validators, **attrs):
39.238 + self.args = args
39.239 + super(Dropdown, self).__init__(name, *validators, **attrs)
39.240 +
39.241 + def render(self):
39.242 + attrs = self.attrs.copy()
39.243 + attrs['name'] = self.name
39.244 +
39.245 + x = '<select %s>\n' % attrs
39.246 +
39.247 + for arg in self.args:
39.248 + x += self._render_option(arg)
39.249 +
39.250 + x += '</select>\n'
39.251 + return x
39.252 +
39.253 + def _render_option(self, arg, indent=' '):
39.254 + if isinstance(arg, (tuple, list)):
39.255 + value, desc= arg
39.256 + else:
39.257 + value, desc = arg, arg
39.258 +
39.259 + if self.value == value or (isinstance(self.value, list) and value in self.value):
39.260 + select_p = ' selected="selected"'
39.261 + else:
39.262 + select_p = ''
39.263 + return indent + '<option%s value="%s">%s</option>\n' % (select_p, net.websafe(value), net.websafe(desc))
39.264 +
39.265 +
39.266 +class GroupedDropdown(Dropdown):
39.267 + r"""Grouped Dropdown/select input.
39.268 +
39.269 + >>> GroupedDropdown(name='car_type', args=(('Swedish Cars', ('Volvo', 'Saab')), ('German Cars', ('Mercedes', 'Audi'))), value='Audi').render()
39.270 + u'<select id="car_type" name="car_type">\n <optgroup label="Swedish Cars">\n <option value="Volvo">Volvo</option>\n <option value="Saab">Saab</option>\n </optgroup>\n <optgroup label="German Cars">\n <option value="Mercedes">Mercedes</option>\n <option selected="selected" value="Audi">Audi</option>\n </optgroup>\n</select>\n'
39.271 + >>> GroupedDropdown(name='car_type', args=(('Swedish Cars', (('v', 'Volvo'), ('s', 'Saab'))), ('German Cars', (('m', 'Mercedes'), ('a', 'Audi')))), value='a').render()
39.272 + u'<select id="car_type" name="car_type">\n <optgroup label="Swedish Cars">\n <option value="v">Volvo</option>\n <option value="s">Saab</option>\n </optgroup>\n <optgroup label="German Cars">\n <option value="m">Mercedes</option>\n <option selected="selected" value="a">Audi</option>\n </optgroup>\n</select>\n'
39.273 +
39.274 + """
39.275 + def __init__(self, name, args, *validators, **attrs):
39.276 + self.args = args
39.277 + super(Dropdown, self).__init__(name, *validators, **attrs)
39.278 +
39.279 + def render(self):
39.280 + attrs = self.attrs.copy()
39.281 + attrs['name'] = self.name
39.282 +
39.283 + x = '<select %s>\n' % attrs
39.284 +
39.285 + for label, options in self.args:
39.286 + x += ' <optgroup label="%s">\n' % net.websafe(label)
39.287 + for arg in options:
39.288 + x += self._render_option(arg, indent = ' ')
39.289 + x += ' </optgroup>\n'
39.290 +
39.291 + x += '</select>\n'
39.292 + return x
39.293 +
39.294 +class Radio(Input):
39.295 + def __init__(self, name, args, *validators, **attrs):
39.296 + self.args = args
39.297 + super(Radio, self).__init__(name, *validators, **attrs)
39.298 +
39.299 + def render(self):
39.300 + x = '<span>'
39.301 + for arg in self.args:
39.302 + if isinstance(arg, (tuple, list)):
39.303 + value, desc= arg
39.304 + else:
39.305 + value, desc = arg, arg
39.306 + attrs = self.attrs.copy()
39.307 + attrs['name'] = self.name
39.308 + attrs['type'] = 'radio'
39.309 + attrs['value'] = value
39.310 + if self.value == value:
39.311 + attrs['checked'] = 'checked'
39.312 + x += '<input %s/> %s' % (attrs, net.websafe(desc))
39.313 + x += '</span>'
39.314 + return x
39.315 +
39.316 +class Checkbox(Input):
39.317 + """Checkbox input.
39.318 +
39.319 + >>> Checkbox('foo', value='bar', checked=True).render()
39.320 + u'<input checked="checked" type="checkbox" id="foo_bar" value="bar" name="foo"/>'
39.321 + >>> Checkbox('foo', value='bar').render()
39.322 + u'<input type="checkbox" id="foo_bar" value="bar" name="foo"/>'
39.323 + >>> c = Checkbox('foo', value='bar')
39.324 + >>> c.validate('on')
39.325 + True
39.326 + >>> c.render()
39.327 + u'<input checked="checked" type="checkbox" id="foo_bar" value="bar" name="foo"/>'
39.328 + """
39.329 + def __init__(self, name, *validators, **attrs):
39.330 + self.checked = attrs.pop('checked', False)
39.331 + Input.__init__(self, name, *validators, **attrs)
39.332 +
39.333 + def get_default_id(self):
39.334 + value = utils.safestr(self.value or "")
39.335 + return self.name + '_' + value.replace(' ', '_')
39.336 +
39.337 + def render(self):
39.338 + attrs = self.attrs.copy()
39.339 + attrs['type'] = 'checkbox'
39.340 + attrs['name'] = self.name
39.341 + attrs['value'] = self.value
39.342 +
39.343 + if self.checked:
39.344 + attrs['checked'] = 'checked'
39.345 + return '<input %s/>' % attrs
39.346 +
39.347 + def set_value(self, value):
39.348 + self.checked = bool(value)
39.349 +
39.350 + def get_value(self):
39.351 + return self.checked
39.352 +
39.353 +class Button(Input):
39.354 + """HTML Button.
39.355 +
39.356 + >>> Button("save").render()
39.357 + u'<button id="save" name="save">save</button>'
39.358 + >>> Button("action", value="save", html="<b>Save Changes</b>").render()
39.359 + u'<button id="action" value="save" name="action"><b>Save Changes</b></button>'
39.360 + """
39.361 + def __init__(self, name, *validators, **attrs):
39.362 + super(Button, self).__init__(name, *validators, **attrs)
39.363 + self.description = ""
39.364 +
39.365 + def render(self):
39.366 + attrs = self.attrs.copy()
39.367 + attrs['name'] = self.name
39.368 + if self.value is not None:
39.369 + attrs['value'] = self.value
39.370 + html = attrs.pop('html', None) or net.websafe(self.name)
39.371 + return '<button %s>%s</button>' % (attrs, html)
39.372 +
39.373 +class Hidden(Input):
39.374 + """Hidden Input.
39.375 +
39.376 + >>> Hidden(name='foo', value='bar').render()
39.377 + u'<input type="hidden" id="foo" value="bar" name="foo"/>'
39.378 + """
39.379 + def is_hidden(self):
39.380 + return True
39.381 +
39.382 + def get_type(self):
39.383 + return 'hidden'
39.384 +
39.385 +class File(Input):
39.386 + """File input.
39.387 +
39.388 + >>> File(name='f').render()
39.389 + u'<input type="file" id="f" name="f"/>'
39.390 + """
39.391 + def get_type(self):
39.392 + return 'file'
39.393 +
39.394 +class Validator:
39.395 + def __deepcopy__(self, memo): return copy.copy(self)
39.396 + def __init__(self, msg, test, jstest=None): utils.autoassign(self, locals())
39.397 + def valid(self, value):
39.398 + try: return self.test(value)
39.399 + except: return False
39.400 +
39.401 +notnull = Validator("Required", bool)
39.402 +
39.403 +class regexp(Validator):
39.404 + def __init__(self, rexp, msg):
39.405 + self.rexp = re.compile(rexp)
39.406 + self.msg = msg
39.407 +
39.408 + def valid(self, value):
39.409 + return bool(self.rexp.match(value))
39.410 +
39.411 +if __name__ == "__main__":
39.412 + import doctest
39.413 + doctest.testmod()
40.1 --- /dev/null Thu Jan 01 00:00:00 1970 +0000
40.2 +++ b/OpenSecurity/install/web.py-0.37/build/lib/web/http.py Mon Dec 02 14:02:05 2013 +0100
40.3 @@ -0,0 +1,150 @@
40.4 +"""
40.5 +HTTP Utilities
40.6 +(from web.py)
40.7 +"""
40.8 +
40.9 +__all__ = [
40.10 + "expires", "lastmodified",
40.11 + "prefixurl", "modified",
40.12 + "changequery", "url",
40.13 + "profiler",
40.14 +]
40.15 +
40.16 +import sys, os, threading, urllib, urlparse
40.17 +try: import datetime
40.18 +except ImportError: pass
40.19 +import net, utils, webapi as web
40.20 +
40.21 +def prefixurl(base=''):
40.22 + """
40.23 + Sorry, this function is really difficult to explain.
40.24 + Maybe some other time.
40.25 + """
40.26 + url = web.ctx.path.lstrip('/')
40.27 + for i in xrange(url.count('/')):
40.28 + base += '../'
40.29 + if not base:
40.30 + base = './'
40.31 + return base
40.32 +
40.33 +def expires(delta):
40.34 + """
40.35 + Outputs an `Expires` header for `delta` from now.
40.36 + `delta` is a `timedelta` object or a number of seconds.
40.37 + """
40.38 + if isinstance(delta, (int, long)):
40.39 + delta = datetime.timedelta(seconds=delta)
40.40 + date_obj = datetime.datetime.utcnow() + delta
40.41 + web.header('Expires', net.httpdate(date_obj))
40.42 +
40.43 +def lastmodified(date_obj):
40.44 + """Outputs a `Last-Modified` header for `datetime`."""
40.45 + web.header('Last-Modified', net.httpdate(date_obj))
40.46 +
40.47 +def modified(date=None, etag=None):
40.48 + """
40.49 + Checks to see if the page has been modified since the version in the
40.50 + requester's cache.
40.51 +
40.52 + When you publish pages, you can include `Last-Modified` and `ETag`
40.53 + with the date the page was last modified and an opaque token for
40.54 + the particular version, respectively. When readers reload the page,
40.55 + the browser sends along the modification date and etag value for
40.56 + the version it has in its cache. If the page hasn't changed,
40.57 + the server can just return `304 Not Modified` and not have to
40.58 + send the whole page again.
40.59 +
40.60 + This function takes the last-modified date `date` and the ETag `etag`
40.61 + and checks the headers to see if they match. If they do, it returns
40.62 + `True`, or otherwise it raises NotModified error. It also sets
40.63 + `Last-Modified` and `ETag` output headers.
40.64 + """
40.65 + try:
40.66 + from __builtin__ import set
40.67 + except ImportError:
40.68 + # for python 2.3
40.69 + from sets import Set as set
40.70 +
40.71 + n = set([x.strip('" ') for x in web.ctx.env.get('HTTP_IF_NONE_MATCH', '').split(',')])
40.72 + m = net.parsehttpdate(web.ctx.env.get('HTTP_IF_MODIFIED_SINCE', '').split(';')[0])
40.73 + validate = False
40.74 + if etag:
40.75 + if '*' in n or etag in n:
40.76 + validate = True
40.77 + if date and m:
40.78 + # we subtract a second because
40.79 + # HTTP dates don't have sub-second precision
40.80 + if date-datetime.timedelta(seconds=1) <= m:
40.81 + validate = True
40.82 +
40.83 + if date: lastmodified(date)
40.84 + if etag: web.header('ETag', '"' + etag + '"')
40.85 + if validate:
40.86 + raise web.notmodified()
40.87 + else:
40.88 + return True
40.89 +
40.90 +def urlencode(query, doseq=0):
40.91 + """
40.92 + Same as urllib.urlencode, but supports unicode strings.
40.93 +
40.94 + >>> urlencode({'text':'foo bar'})
40.95 + 'text=foo+bar'
40.96 + >>> urlencode({'x': [1, 2]}, doseq=True)
40.97 + 'x=1&x=2'
40.98 + """
40.99 + def convert(value, doseq=False):
40.100 + if doseq and isinstance(value, list):
40.101 + return [convert(v) for v in value]
40.102 + else:
40.103 + return utils.safestr(value)
40.104 +
40.105 + query = dict([(k, convert(v, doseq)) for k, v in query.items()])
40.106 + return urllib.urlencode(query, doseq=doseq)
40.107 +
40.108 +def changequery(query=None, **kw):
40.109 + """
40.110 + Imagine you're at `/foo?a=1&b=2`. Then `changequery(a=3)` will return
40.111 + `/foo?a=3&b=2` -- the same URL but with the arguments you requested
40.112 + changed.
40.113 + """
40.114 + if query is None:
40.115 + query = web.rawinput(method='get')
40.116 + for k, v in kw.iteritems():
40.117 + if v is None:
40.118 + query.pop(k, None)
40.119 + else:
40.120 + query[k] = v
40.121 + out = web.ctx.path
40.122 + if query:
40.123 + out += '?' + urlencode(query, doseq=True)
40.124 + return out
40.125 +
40.126 +def url(path=None, doseq=False, **kw):
40.127 + """
40.128 + Makes url by concatenating web.ctx.homepath and path and the
40.129 + query string created using the arguments.
40.130 + """
40.131 + if path is None:
40.132 + path = web.ctx.path
40.133 + if path.startswith("/"):
40.134 + out = web.ctx.homepath + path
40.135 + else:
40.136 + out = path
40.137 +
40.138 + if kw:
40.139 + out += '?' + urlencode(kw, doseq=doseq)
40.140 +
40.141 + return out
40.142 +
40.143 +def profiler(app):
40.144 + """Outputs basic profiling information at the bottom of each response."""
40.145 + from utils import profile
40.146 + def profile_internal(e, o):
40.147 + out, result = profile(app)(e, o)
40.148 + return list(out) + ['<pre>' + net.websafe(result) + '</pre>']
40.149 + return profile_internal
40.150 +
40.151 +if __name__ == "__main__":
40.152 + import doctest
40.153 + doctest.testmod()
41.1 --- /dev/null Thu Jan 01 00:00:00 1970 +0000
41.2 +++ b/OpenSecurity/install/web.py-0.37/build/lib/web/httpserver.py Mon Dec 02 14:02:05 2013 +0100
41.3 @@ -0,0 +1,319 @@
41.4 +__all__ = ["runsimple"]
41.5 +
41.6 +import sys, os
41.7 +from SimpleHTTPServer import SimpleHTTPRequestHandler
41.8 +import urllib
41.9 +import posixpath
41.10 +
41.11 +import webapi as web
41.12 +import net
41.13 +import utils
41.14 +
41.15 +def runbasic(func, server_address=("0.0.0.0", 8080)):
41.16 + """
41.17 + Runs a simple HTTP server hosting WSGI app `func`. The directory `static/`
41.18 + is hosted statically.
41.19 +
41.20 + Based on [WsgiServer][ws] from [Colin Stewart][cs].
41.21 +
41.22 + [ws]: http://www.owlfish.com/software/wsgiutils/documentation/wsgi-server-api.html
41.23 + [cs]: http://www.owlfish.com/
41.24 + """
41.25 + # Copyright (c) 2004 Colin Stewart (http://www.owlfish.com/)
41.26 + # Modified somewhat for simplicity
41.27 + # Used under the modified BSD license:
41.28 + # http://www.xfree86.org/3.3.6/COPYRIGHT2.html#5
41.29 +
41.30 + import SimpleHTTPServer, SocketServer, BaseHTTPServer, urlparse
41.31 + import socket, errno
41.32 + import traceback
41.33 +
41.34 + class WSGIHandler(SimpleHTTPServer.SimpleHTTPRequestHandler):
41.35 + def run_wsgi_app(self):
41.36 + protocol, host, path, parameters, query, fragment = \
41.37 + urlparse.urlparse('http://dummyhost%s' % self.path)
41.38 +
41.39 + # we only use path, query
41.40 + env = {'wsgi.version': (1, 0)
41.41 + ,'wsgi.url_scheme': 'http'
41.42 + ,'wsgi.input': self.rfile
41.43 + ,'wsgi.errors': sys.stderr
41.44 + ,'wsgi.multithread': 1
41.45 + ,'wsgi.multiprocess': 0
41.46 + ,'wsgi.run_once': 0
41.47 + ,'REQUEST_METHOD': self.command
41.48 + ,'REQUEST_URI': self.path
41.49 + ,'PATH_INFO': path
41.50 + ,'QUERY_STRING': query
41.51 + ,'CONTENT_TYPE': self.headers.get('Content-Type', '')
41.52 + ,'CONTENT_LENGTH': self.headers.get('Content-Length', '')
41.53 + ,'REMOTE_ADDR': self.client_address[0]
41.54 + ,'SERVER_NAME': self.server.server_address[0]
41.55 + ,'SERVER_PORT': str(self.server.server_address[1])
41.56 + ,'SERVER_PROTOCOL': self.request_version
41.57 + }
41.58 +
41.59 + for http_header, http_value in self.headers.items():
41.60 + env ['HTTP_%s' % http_header.replace('-', '_').upper()] = \
41.61 + http_value
41.62 +
41.63 + # Setup the state
41.64 + self.wsgi_sent_headers = 0
41.65 + self.wsgi_headers = []
41.66 +
41.67 + try:
41.68 + # We have there environment, now invoke the application
41.69 + result = self.server.app(env, self.wsgi_start_response)
41.70 + try:
41.71 + try:
41.72 + for data in result:
41.73 + if data:
41.74 + self.wsgi_write_data(data)
41.75 + finally:
41.76 + if hasattr(result, 'close'):
41.77 + result.close()
41.78 + except socket.error, socket_err:
41.79 + # Catch common network errors and suppress them
41.80 + if (socket_err.args[0] in \
41.81 + (errno.ECONNABORTED, errno.EPIPE)):
41.82 + return
41.83 + except socket.timeout, socket_timeout:
41.84 + return
41.85 + except:
41.86 + print >> web.debug, traceback.format_exc(),
41.87 +
41.88 + if (not self.wsgi_sent_headers):
41.89 + # We must write out something!
41.90 + self.wsgi_write_data(" ")
41.91 + return
41.92 +
41.93 + do_POST = run_wsgi_app
41.94 + do_PUT = run_wsgi_app
41.95 + do_DELETE = run_wsgi_app
41.96 +
41.97 + def do_GET(self):
41.98 + if self.path.startswith('/static/'):
41.99 + SimpleHTTPServer.SimpleHTTPRequestHandler.do_GET(self)
41.100 + else:
41.101 + self.run_wsgi_app()
41.102 +
41.103 + def wsgi_start_response(self, response_status, response_headers,
41.104 + exc_info=None):
41.105 + if (self.wsgi_sent_headers):
41.106 + raise Exception \
41.107 + ("Headers already sent and start_response called again!")
41.108 + # Should really take a copy to avoid changes in the application....
41.109 + self.wsgi_headers = (response_status, response_headers)
41.110 + return self.wsgi_write_data
41.111 +
41.112 + def wsgi_write_data(self, data):
41.113 + if (not self.wsgi_sent_headers):
41.114 + status, headers = self.wsgi_headers
41.115 + # Need to send header prior to data
41.116 + status_code = status[:status.find(' ')]
41.117 + status_msg = status[status.find(' ') + 1:]
41.118 + self.send_response(int(status_code), status_msg)
41.119 + for header, value in headers:
41.120 + self.send_header(header, value)
41.121 + self.end_headers()
41.122 + self.wsgi_sent_headers = 1
41.123 + # Send the data
41.124 + self.wfile.write(data)
41.125 +
41.126 + class WSGIServer(SocketServer.ThreadingMixIn, BaseHTTPServer.HTTPServer):
41.127 + def __init__(self, func, server_address):
41.128 + BaseHTTPServer.HTTPServer.__init__(self,
41.129 + server_address,
41.130 + WSGIHandler)
41.131 + self.app = func
41.132 + self.serverShuttingDown = 0
41.133 +
41.134 + print "http://%s:%d/" % server_address
41.135 + WSGIServer(func, server_address).serve_forever()
41.136 +
41.137 +# The WSGIServer instance.
41.138 +# Made global so that it can be stopped in embedded mode.
41.139 +server = None
41.140 +
41.141 +def runsimple(func, server_address=("0.0.0.0", 8080)):
41.142 + """
41.143 + Runs [CherryPy][cp] WSGI server hosting WSGI app `func`.
41.144 + The directory `static/` is hosted statically.
41.145 +
41.146 + [cp]: http://www.cherrypy.org
41.147 + """
41.148 + global server
41.149 + func = StaticMiddleware(func)
41.150 + func = LogMiddleware(func)
41.151 +
41.152 + server = WSGIServer(server_address, func)
41.153 +
41.154 + if server.ssl_adapter:
41.155 + print "https://%s:%d/" % server_address
41.156 + else:
41.157 + print "http://%s:%d/" % server_address
41.158 +
41.159 + try:
41.160 + server.start()
41.161 + except (KeyboardInterrupt, SystemExit):
41.162 + server.stop()
41.163 + server = None
41.164 +
41.165 +def WSGIServer(server_address, wsgi_app):
41.166 + """Creates CherryPy WSGI server listening at `server_address` to serve `wsgi_app`.
41.167 + This function can be overwritten to customize the webserver or use a different webserver.
41.168 + """
41.169 + import wsgiserver
41.170 +
41.171 + # Default values of wsgiserver.ssl_adapters uses cherrypy.wsgiserver
41.172 + # prefix. Overwriting it make it work with web.wsgiserver.
41.173 + wsgiserver.ssl_adapters = {
41.174 + 'builtin': 'web.wsgiserver.ssl_builtin.BuiltinSSLAdapter',
41.175 + 'pyopenssl': 'web.wsgiserver.ssl_pyopenssl.pyOpenSSLAdapter',
41.176 + }
41.177 +
41.178 + server = wsgiserver.CherryPyWSGIServer(server_address, wsgi_app, server_name="localhost")
41.179 +
41.180 + def create_ssl_adapter(cert, key):
41.181 + # wsgiserver tries to import submodules as cherrypy.wsgiserver.foo.
41.182 + # That doesn't work as not it is web.wsgiserver.
41.183 + # Patching sys.modules temporarily to make it work.
41.184 + import types
41.185 + cherrypy = types.ModuleType('cherrypy')
41.186 + cherrypy.wsgiserver = wsgiserver
41.187 + sys.modules['cherrypy'] = cherrypy
41.188 + sys.modules['cherrypy.wsgiserver'] = wsgiserver
41.189 +
41.190 + from wsgiserver.ssl_pyopenssl import pyOpenSSLAdapter
41.191 + adapter = pyOpenSSLAdapter(cert, key)
41.192 +
41.193 + # We are done with our work. Cleanup the patches.
41.194 + del sys.modules['cherrypy']
41.195 + del sys.modules['cherrypy.wsgiserver']
41.196 +
41.197 + return adapter
41.198 +
41.199 + # SSL backward compatibility
41.200 + if (server.ssl_adapter is None and
41.201 + getattr(server, 'ssl_certificate', None) and
41.202 + getattr(server, 'ssl_private_key', None)):
41.203 + server.ssl_adapter = create_ssl_adapter(server.ssl_certificate, server.ssl_private_key)
41.204 +
41.205 + server.nodelay = not sys.platform.startswith('java') # TCP_NODELAY isn't supported on the JVM
41.206 + return server
41.207 +
41.208 +class StaticApp(SimpleHTTPRequestHandler):
41.209 + """WSGI application for serving static files."""
41.210 + def __init__(self, environ, start_response):
41.211 + self.headers = []
41.212 + self.environ = environ
41.213 + self.start_response = start_response
41.214 +
41.215 + def send_response(self, status, msg=""):
41.216 + self.status = str(status) + " " + msg
41.217 +
41.218 + def send_header(self, name, value):
41.219 + self.headers.append((name, value))
41.220 +
41.221 + def end_headers(self):
41.222 + pass
41.223 +
41.224 + def log_message(*a): pass
41.225 +
41.226 + def __iter__(self):
41.227 + environ = self.environ
41.228 +
41.229 + self.path = environ.get('PATH_INFO', '')
41.230 + self.client_address = environ.get('REMOTE_ADDR','-'), \
41.231 + environ.get('REMOTE_PORT','-')
41.232 + self.command = environ.get('REQUEST_METHOD', '-')
41.233 +
41.234 + from cStringIO import StringIO
41.235 + self.wfile = StringIO() # for capturing error
41.236 +
41.237 + try:
41.238 + path = self.translate_path(self.path)
41.239 + etag = '"%s"' % os.path.getmtime(path)
41.240 + client_etag = environ.get('HTTP_IF_NONE_MATCH')
41.241 + self.send_header('ETag', etag)
41.242 + if etag == client_etag:
41.243 + self.send_response(304, "Not Modified")
41.244 + self.start_response(self.status, self.headers)
41.245 + raise StopIteration
41.246 + except OSError:
41.247 + pass # Probably a 404
41.248 +
41.249 + f = self.send_head()
41.250 + self.start_response(self.status, self.headers)
41.251 +
41.252 + if f:
41.253 + block_size = 16 * 1024
41.254 + while True:
41.255 + buf = f.read(block_size)
41.256 + if not buf:
41.257 + break
41.258 + yield buf
41.259 + f.close()
41.260 + else:
41.261 + value = self.wfile.getvalue()
41.262 + yield value
41.263 +
41.264 +class StaticMiddleware:
41.265 + """WSGI middleware for serving static files."""
41.266 + def __init__(self, app, prefix='/static/'):
41.267 + self.app = app
41.268 + self.prefix = prefix
41.269 +
41.270 + def __call__(self, environ, start_response):
41.271 + path = environ.get('PATH_INFO', '')
41.272 + path = self.normpath(path)
41.273 +
41.274 + if path.startswith(self.prefix):
41.275 + return StaticApp(environ, start_response)
41.276 + else:
41.277 + return self.app(environ, start_response)
41.278 +
41.279 + def normpath(self, path):
41.280 + path2 = posixpath.normpath(urllib.unquote(path))
41.281 + if path.endswith("/"):
41.282 + path2 += "/"
41.283 + return path2
41.284 +
41.285 +
41.286 +class LogMiddleware:
41.287 + """WSGI middleware for logging the status."""
41.288 + def __init__(self, app):
41.289 + self.app = app
41.290 + self.format = '%s - - [%s] "%s %s %s" - %s'
41.291 +
41.292 + from BaseHTTPServer import BaseHTTPRequestHandler
41.293 + import StringIO
41.294 + f = StringIO.StringIO()
41.295 +
41.296 + class FakeSocket:
41.297 + def makefile(self, *a):
41.298 + return f
41.299 +
41.300 + # take log_date_time_string method from BaseHTTPRequestHandler
41.301 + self.log_date_time_string = BaseHTTPRequestHandler(FakeSocket(), None, None).log_date_time_string
41.302 +
41.303 + def __call__(self, environ, start_response):
41.304 + def xstart_response(status, response_headers, *args):
41.305 + out = start_response(status, response_headers, *args)
41.306 + self.log(status, environ)
41.307 + return out
41.308 +
41.309 + return self.app(environ, xstart_response)
41.310 +
41.311 + def log(self, status, environ):
41.312 + outfile = environ.get('wsgi.errors', web.debug)
41.313 + req = environ.get('PATH_INFO', '_')
41.314 + protocol = environ.get('ACTUAL_SERVER_PROTOCOL', '-')
41.315 + method = environ.get('REQUEST_METHOD', '-')
41.316 + host = "%s:%s" % (environ.get('REMOTE_ADDR','-'),
41.317 + environ.get('REMOTE_PORT','-'))
41.318 +
41.319 + time = self.log_date_time_string()
41.320 +
41.321 + msg = self.format % (host, time, protocol, method, req, status)
41.322 + print >> outfile, utils.safestr(msg)
42.1 --- /dev/null Thu Jan 01 00:00:00 1970 +0000
42.2 +++ b/OpenSecurity/install/web.py-0.37/build/lib/web/net.py Mon Dec 02 14:02:05 2013 +0100
42.3 @@ -0,0 +1,193 @@
42.4 +"""
42.5 +Network Utilities
42.6 +(from web.py)
42.7 +"""
42.8 +
42.9 +__all__ = [
42.10 + "validipaddr", "validipport", "validip", "validaddr",
42.11 + "urlquote",
42.12 + "httpdate", "parsehttpdate",
42.13 + "htmlquote", "htmlunquote", "websafe",
42.14 +]
42.15 +
42.16 +import urllib, time
42.17 +try: import datetime
42.18 +except ImportError: pass
42.19 +
42.20 +def validipaddr(address):
42.21 + """
42.22 + Returns True if `address` is a valid IPv4 address.
42.23 +
42.24 + >>> validipaddr('192.168.1.1')
42.25 + True
42.26 + >>> validipaddr('192.168.1.800')
42.27 + False
42.28 + >>> validipaddr('192.168.1')
42.29 + False
42.30 + """
42.31 + try:
42.32 + octets = address.split('.')
42.33 + if len(octets) != 4:
42.34 + return False
42.35 + for x in octets:
42.36 + if not (0 <= int(x) <= 255):
42.37 + return False
42.38 + except ValueError:
42.39 + return False
42.40 + return True
42.41 +
42.42 +def validipport(port):
42.43 + """
42.44 + Returns True if `port` is a valid IPv4 port.
42.45 +
42.46 + >>> validipport('9000')
42.47 + True
42.48 + >>> validipport('foo')
42.49 + False
42.50 + >>> validipport('1000000')
42.51 + False
42.52 + """
42.53 + try:
42.54 + if not (0 <= int(port) <= 65535):
42.55 + return False
42.56 + except ValueError:
42.57 + return False
42.58 + return True
42.59 +
42.60 +def validip(ip, defaultaddr="0.0.0.0", defaultport=8080):
42.61 + """Returns `(ip_address, port)` from string `ip_addr_port`"""
42.62 + addr = defaultaddr
42.63 + port = defaultport
42.64 +
42.65 + ip = ip.split(":", 1)
42.66 + if len(ip) == 1:
42.67 + if not ip[0]:
42.68 + pass
42.69 + elif validipaddr(ip[0]):
42.70 + addr = ip[0]
42.71 + elif validipport(ip[0]):
42.72 + port = int(ip[0])
42.73 + else:
42.74 + raise ValueError, ':'.join(ip) + ' is not a valid IP address/port'
42.75 + elif len(ip) == 2:
42.76 + addr, port = ip
42.77 + if not validipaddr(addr) and validipport(port):
42.78 + raise ValueError, ':'.join(ip) + ' is not a valid IP address/port'
42.79 + port = int(port)
42.80 + else:
42.81 + raise ValueError, ':'.join(ip) + ' is not a valid IP address/port'
42.82 + return (addr, port)
42.83 +
42.84 +def validaddr(string_):
42.85 + """
42.86 + Returns either (ip_address, port) or "/path/to/socket" from string_
42.87 +
42.88 + >>> validaddr('/path/to/socket')
42.89 + '/path/to/socket'
42.90 + >>> validaddr('8000')
42.91 + ('0.0.0.0', 8000)
42.92 + >>> validaddr('127.0.0.1')
42.93 + ('127.0.0.1', 8080)
42.94 + >>> validaddr('127.0.0.1:8000')
42.95 + ('127.0.0.1', 8000)
42.96 + >>> validaddr('fff')
42.97 + Traceback (most recent call last):
42.98 + ...
42.99 + ValueError: fff is not a valid IP address/port
42.100 + """
42.101 + if '/' in string_:
42.102 + return string_
42.103 + else:
42.104 + return validip(string_)
42.105 +
42.106 +def urlquote(val):
42.107 + """
42.108 + Quotes a string for use in a URL.
42.109 +
42.110 + >>> urlquote('://?f=1&j=1')
42.111 + '%3A//%3Ff%3D1%26j%3D1'
42.112 + >>> urlquote(None)
42.113 + ''
42.114 + >>> urlquote(u'\u203d')
42.115 + '%E2%80%BD'
42.116 + """
42.117 + if val is None: return ''
42.118 + if not isinstance(val, unicode): val = str(val)
42.119 + else: val = val.encode('utf-8')
42.120 + return urllib.quote(val)
42.121 +
42.122 +def httpdate(date_obj):
42.123 + """
42.124 + Formats a datetime object for use in HTTP headers.
42.125 +
42.126 + >>> import datetime
42.127 + >>> httpdate(datetime.datetime(1970, 1, 1, 1, 1, 1))
42.128 + 'Thu, 01 Jan 1970 01:01:01 GMT'
42.129 + """
42.130 + return date_obj.strftime("%a, %d %b %Y %H:%M:%S GMT")
42.131 +
42.132 +def parsehttpdate(string_):
42.133 + """
42.134 + Parses an HTTP date into a datetime object.
42.135 +
42.136 + >>> parsehttpdate('Thu, 01 Jan 1970 01:01:01 GMT')
42.137 + datetime.datetime(1970, 1, 1, 1, 1, 1)
42.138 + """
42.139 + try:
42.140 + t = time.strptime(string_, "%a, %d %b %Y %H:%M:%S %Z")
42.141 + except ValueError:
42.142 + return None
42.143 + return datetime.datetime(*t[:6])
42.144 +
42.145 +def htmlquote(text):
42.146 + r"""
42.147 + Encodes `text` for raw use in HTML.
42.148 +
42.149 + >>> htmlquote(u"<'&\">")
42.150 + u'<'&">'
42.151 + """
42.152 + text = text.replace(u"&", u"&") # Must be done first!
42.153 + text = text.replace(u"<", u"<")
42.154 + text = text.replace(u">", u">")
42.155 + text = text.replace(u"'", u"'")
42.156 + text = text.replace(u'"', u""")
42.157 + return text
42.158 +
42.159 +def htmlunquote(text):
42.160 + r"""
42.161 + Decodes `text` that's HTML quoted.
42.162 +
42.163 + >>> htmlunquote(u'<'&">')
42.164 + u'<\'&">'
42.165 + """
42.166 + text = text.replace(u""", u'"')
42.167 + text = text.replace(u"'", u"'")
42.168 + text = text.replace(u">", u">")
42.169 + text = text.replace(u"<", u"<")
42.170 + text = text.replace(u"&", u"&") # Must be done last!
42.171 + return text
42.172 +
42.173 +def websafe(val):
42.174 + r"""Converts `val` so that it is safe for use in Unicode HTML.
42.175 +
42.176 + >>> websafe("<'&\">")
42.177 + u'<'&">'
42.178 + >>> websafe(None)
42.179 + u''
42.180 + >>> websafe(u'\u203d')
42.181 + u'\u203d'
42.182 + >>> websafe('\xe2\x80\xbd')
42.183 + u'\u203d'
42.184 + """
42.185 + if val is None:
42.186 + return u''
42.187 + elif isinstance(val, str):
42.188 + val = val.decode('utf-8')
42.189 + elif not isinstance(val, unicode):
42.190 + val = unicode(val)
42.191 +
42.192 + return htmlquote(val)
42.193 +
42.194 +if __name__ == "__main__":
42.195 + import doctest
42.196 + doctest.testmod()
43.1 --- /dev/null Thu Jan 01 00:00:00 1970 +0000
43.2 +++ b/OpenSecurity/install/web.py-0.37/build/lib/web/python23.py Mon Dec 02 14:02:05 2013 +0100
43.3 @@ -0,0 +1,46 @@
43.4 +"""Python 2.3 compatabilty"""
43.5 +import threading
43.6 +
43.7 +class threadlocal(object):
43.8 + """Implementation of threading.local for python2.3.
43.9 + """
43.10 + def __getattribute__(self, name):
43.11 + if name == "__dict__":
43.12 + return threadlocal._getd(self)
43.13 + else:
43.14 + try:
43.15 + return object.__getattribute__(self, name)
43.16 + except AttributeError:
43.17 + try:
43.18 + return self.__dict__[name]
43.19 + except KeyError:
43.20 + raise AttributeError, name
43.21 +
43.22 + def __setattr__(self, name, value):
43.23 + self.__dict__[name] = value
43.24 +
43.25 + def __delattr__(self, name):
43.26 + try:
43.27 + del self.__dict__[name]
43.28 + except KeyError:
43.29 + raise AttributeError, name
43.30 +
43.31 + def _getd(self):
43.32 + t = threading.currentThread()
43.33 + if not hasattr(t, '_d'):
43.34 + # using __dict__ of thread as thread local storage
43.35 + t._d = {}
43.36 +
43.37 + _id = id(self)
43.38 + # there could be multiple instances of threadlocal.
43.39 + # use id(self) as key
43.40 + if _id not in t._d:
43.41 + t._d[_id] = {}
43.42 + return t._d[_id]
43.43 +
43.44 +if __name__ == '__main__':
43.45 + d = threadlocal()
43.46 + d.x = 1
43.47 + print d.__dict__
43.48 + print d.x
43.49 +
43.50 \ No newline at end of file
44.1 --- /dev/null Thu Jan 01 00:00:00 1970 +0000
44.2 +++ b/OpenSecurity/install/web.py-0.37/build/lib/web/session.py Mon Dec 02 14:02:05 2013 +0100
44.3 @@ -0,0 +1,358 @@
44.4 +"""
44.5 +Session Management
44.6 +(from web.py)
44.7 +"""
44.8 +
44.9 +import os, time, datetime, random, base64
44.10 +import os.path
44.11 +from copy import deepcopy
44.12 +try:
44.13 + import cPickle as pickle
44.14 +except ImportError:
44.15 + import pickle
44.16 +try:
44.17 + import hashlib
44.18 + sha1 = hashlib.sha1
44.19 +except ImportError:
44.20 + import sha
44.21 + sha1 = sha.new
44.22 +
44.23 +import utils
44.24 +import webapi as web
44.25 +
44.26 +__all__ = [
44.27 + 'Session', 'SessionExpired',
44.28 + 'Store', 'DiskStore', 'DBStore',
44.29 +]
44.30 +
44.31 +web.config.session_parameters = utils.storage({
44.32 + 'cookie_name': 'webpy_session_id',
44.33 + 'cookie_domain': None,
44.34 + 'cookie_path' : None,
44.35 + 'timeout': 86400, #24 * 60 * 60, # 24 hours in seconds
44.36 + 'ignore_expiry': True,
44.37 + 'ignore_change_ip': True,
44.38 + 'secret_key': 'fLjUfxqXtfNoIldA0A0J',
44.39 + 'expired_message': 'Session expired',
44.40 + 'httponly': True,
44.41 + 'secure': False
44.42 +})
44.43 +
44.44 +class SessionExpired(web.HTTPError):
44.45 + def __init__(self, message):
44.46 + web.HTTPError.__init__(self, '200 OK', {}, data=message)
44.47 +
44.48 +class Session(object):
44.49 + """Session management for web.py
44.50 + """
44.51 + __slots__ = [
44.52 + "store", "_initializer", "_last_cleanup_time", "_config", "_data",
44.53 + "__getitem__", "__setitem__", "__delitem__"
44.54 + ]
44.55 +
44.56 + def __init__(self, app, store, initializer=None):
44.57 + self.store = store
44.58 + self._initializer = initializer
44.59 + self._last_cleanup_time = 0
44.60 + self._config = utils.storage(web.config.session_parameters)
44.61 + self._data = utils.threadeddict()
44.62 +
44.63 + self.__getitem__ = self._data.__getitem__
44.64 + self.__setitem__ = self._data.__setitem__
44.65 + self.__delitem__ = self._data.__delitem__
44.66 +
44.67 + if app:
44.68 + app.add_processor(self._processor)
44.69 +
44.70 + def __contains__(self, name):
44.71 + return name in self._data
44.72 +
44.73 + def __getattr__(self, name):
44.74 + return getattr(self._data, name)
44.75 +
44.76 + def __setattr__(self, name, value):
44.77 + if name in self.__slots__:
44.78 + object.__setattr__(self, name, value)
44.79 + else:
44.80 + setattr(self._data, name, value)
44.81 +
44.82 + def __delattr__(self, name):
44.83 + delattr(self._data, name)
44.84 +
44.85 + def _processor(self, handler):
44.86 + """Application processor to setup session for every request"""
44.87 + self._cleanup()
44.88 + self._load()
44.89 +
44.90 + try:
44.91 + return handler()
44.92 + finally:
44.93 + self._save()
44.94 +
44.95 + def _load(self):
44.96 + """Load the session from the store, by the id from cookie"""
44.97 + cookie_name = self._config.cookie_name
44.98 + cookie_domain = self._config.cookie_domain
44.99 + cookie_path = self._config.cookie_path
44.100 + httponly = self._config.httponly
44.101 + self.session_id = web.cookies().get(cookie_name)
44.102 +
44.103 + # protection against session_id tampering
44.104 + if self.session_id and not self._valid_session_id(self.session_id):
44.105 + self.session_id = None
44.106 +
44.107 + self._check_expiry()
44.108 + if self.session_id:
44.109 + d = self.store[self.session_id]
44.110 + self.update(d)
44.111 + self._validate_ip()
44.112 +
44.113 + if not self.session_id:
44.114 + self.session_id = self._generate_session_id()
44.115 +
44.116 + if self._initializer:
44.117 + if isinstance(self._initializer, dict):
44.118 + self.update(deepcopy(self._initializer))
44.119 + elif hasattr(self._initializer, '__call__'):
44.120 + self._initializer()
44.121 +
44.122 + self.ip = web.ctx.ip
44.123 +
44.124 + def _check_expiry(self):
44.125 + # check for expiry
44.126 + if self.session_id and self.session_id not in self.store:
44.127 + if self._config.ignore_expiry:
44.128 + self.session_id = None
44.129 + else:
44.130 + return self.expired()
44.131 +
44.132 + def _validate_ip(self):
44.133 + # check for change of IP
44.134 + if self.session_id and self.get('ip', None) != web.ctx.ip:
44.135 + if not self._config.ignore_change_ip:
44.136 + return self.expired()
44.137 +
44.138 + def _save(self):
44.139 + if not self.get('_killed'):
44.140 + self._setcookie(self.session_id)
44.141 + self.store[self.session_id] = dict(self._data)
44.142 + else:
44.143 + self._setcookie(self.session_id, expires=-1)
44.144 +
44.145 + def _setcookie(self, session_id, expires='', **kw):
44.146 + cookie_name = self._config.cookie_name
44.147 + cookie_domain = self._config.cookie_domain
44.148 + cookie_path = self._config.cookie_path
44.149 + httponly = self._config.httponly
44.150 + secure = self._config.secure
44.151 + web.setcookie(cookie_name, session_id, expires=expires, domain=cookie_domain, httponly=httponly, secure=secure, path=cookie_path)
44.152 +
44.153 + def _generate_session_id(self):
44.154 + """Generate a random id for session"""
44.155 +
44.156 + while True:
44.157 + rand = os.urandom(16)
44.158 + now = time.time()
44.159 + secret_key = self._config.secret_key
44.160 + session_id = sha1("%s%s%s%s" %(rand, now, utils.safestr(web.ctx.ip), secret_key))
44.161 + session_id = session_id.hexdigest()
44.162 + if session_id not in self.store:
44.163 + break
44.164 + return session_id
44.165 +
44.166 + def _valid_session_id(self, session_id):
44.167 + rx = utils.re_compile('^[0-9a-fA-F]+$')
44.168 + return rx.match(session_id)
44.169 +
44.170 + def _cleanup(self):
44.171 + """Cleanup the stored sessions"""
44.172 + current_time = time.time()
44.173 + timeout = self._config.timeout
44.174 + if current_time - self._last_cleanup_time > timeout:
44.175 + self.store.cleanup(timeout)
44.176 + self._last_cleanup_time = current_time
44.177 +
44.178 + def expired(self):
44.179 + """Called when an expired session is atime"""
44.180 + self._killed = True
44.181 + self._save()
44.182 + raise SessionExpired(self._config.expired_message)
44.183 +
44.184 + def kill(self):
44.185 + """Kill the session, make it no longer available"""
44.186 + del self.store[self.session_id]
44.187 + self._killed = True
44.188 +
44.189 +class Store:
44.190 + """Base class for session stores"""
44.191 +
44.192 + def __contains__(self, key):
44.193 + raise NotImplementedError
44.194 +
44.195 + def __getitem__(self, key):
44.196 + raise NotImplementedError
44.197 +
44.198 + def __setitem__(self, key, value):
44.199 + raise NotImplementedError
44.200 +
44.201 + def cleanup(self, timeout):
44.202 + """removes all the expired sessions"""
44.203 + raise NotImplementedError
44.204 +
44.205 + def encode(self, session_dict):
44.206 + """encodes session dict as a string"""
44.207 + pickled = pickle.dumps(session_dict)
44.208 + return base64.encodestring(pickled)
44.209 +
44.210 + def decode(self, session_data):
44.211 + """decodes the data to get back the session dict """
44.212 + pickled = base64.decodestring(session_data)
44.213 + return pickle.loads(pickled)
44.214 +
44.215 +class DiskStore(Store):
44.216 + """
44.217 + Store for saving a session on disk.
44.218 +
44.219 + >>> import tempfile
44.220 + >>> root = tempfile.mkdtemp()
44.221 + >>> s = DiskStore(root)
44.222 + >>> s['a'] = 'foo'
44.223 + >>> s['a']
44.224 + 'foo'
44.225 + >>> time.sleep(0.01)
44.226 + >>> s.cleanup(0.01)
44.227 + >>> s['a']
44.228 + Traceback (most recent call last):
44.229 + ...
44.230 + KeyError: 'a'
44.231 + """
44.232 + def __init__(self, root):
44.233 + # if the storage root doesn't exists, create it.
44.234 + if not os.path.exists(root):
44.235 + os.makedirs(
44.236 + os.path.abspath(root)
44.237 + )
44.238 + self.root = root
44.239 +
44.240 + def _get_path(self, key):
44.241 + if os.path.sep in key:
44.242 + raise ValueError, "Bad key: %s" % repr(key)
44.243 + return os.path.join(self.root, key)
44.244 +
44.245 + def __contains__(self, key):
44.246 + path = self._get_path(key)
44.247 + return os.path.exists(path)
44.248 +
44.249 + def __getitem__(self, key):
44.250 + path = self._get_path(key)
44.251 + if os.path.exists(path):
44.252 + pickled = open(path).read()
44.253 + return self.decode(pickled)
44.254 + else:
44.255 + raise KeyError, key
44.256 +
44.257 + def __setitem__(self, key, value):
44.258 + path = self._get_path(key)
44.259 + pickled = self.encode(value)
44.260 + try:
44.261 + f = open(path, 'w')
44.262 + try:
44.263 + f.write(pickled)
44.264 + finally:
44.265 + f.close()
44.266 + except IOError:
44.267 + pass
44.268 +
44.269 + def __delitem__(self, key):
44.270 + path = self._get_path(key)
44.271 + if os.path.exists(path):
44.272 + os.remove(path)
44.273 +
44.274 + def cleanup(self, timeout):
44.275 + now = time.time()
44.276 + for f in os.listdir(self.root):
44.277 + path = self._get_path(f)
44.278 + atime = os.stat(path).st_atime
44.279 + if now - atime > timeout :
44.280 + os.remove(path)
44.281 +
44.282 +class DBStore(Store):
44.283 + """Store for saving a session in database
44.284 + Needs a table with the following columns:
44.285 +
44.286 + session_id CHAR(128) UNIQUE NOT NULL,
44.287 + atime DATETIME NOT NULL default current_timestamp,
44.288 + data TEXT
44.289 + """
44.290 + def __init__(self, db, table_name):
44.291 + self.db = db
44.292 + self.table = table_name
44.293 +
44.294 + def __contains__(self, key):
44.295 + data = self.db.select(self.table, where="session_id=$key", vars=locals())
44.296 + return bool(list(data))
44.297 +
44.298 + def __getitem__(self, key):
44.299 + now = datetime.datetime.now()
44.300 + try:
44.301 + s = self.db.select(self.table, where="session_id=$key", vars=locals())[0]
44.302 + self.db.update(self.table, where="session_id=$key", atime=now, vars=locals())
44.303 + except IndexError:
44.304 + raise KeyError
44.305 + else:
44.306 + return self.decode(s.data)
44.307 +
44.308 + def __setitem__(self, key, value):
44.309 + pickled = self.encode(value)
44.310 + now = datetime.datetime.now()
44.311 + if key in self:
44.312 + self.db.update(self.table, where="session_id=$key", data=pickled, vars=locals())
44.313 + else:
44.314 + self.db.insert(self.table, False, session_id=key, data=pickled )
44.315 +
44.316 + def __delitem__(self, key):
44.317 + self.db.delete(self.table, where="session_id=$key", vars=locals())
44.318 +
44.319 + def cleanup(self, timeout):
44.320 + timeout = datetime.timedelta(timeout/(24.0*60*60)) #timedelta takes numdays as arg
44.321 + last_allowed_time = datetime.datetime.now() - timeout
44.322 + self.db.delete(self.table, where="$last_allowed_time > atime", vars=locals())
44.323 +
44.324 +class ShelfStore:
44.325 + """Store for saving session using `shelve` module.
44.326 +
44.327 + import shelve
44.328 + store = ShelfStore(shelve.open('session.shelf'))
44.329 +
44.330 + XXX: is shelve thread-safe?
44.331 + """
44.332 + def __init__(self, shelf):
44.333 + self.shelf = shelf
44.334 +
44.335 + def __contains__(self, key):
44.336 + return key in self.shelf
44.337 +
44.338 + def __getitem__(self, key):
44.339 + atime, v = self.shelf[key]
44.340 + self[key] = v # update atime
44.341 + return v
44.342 +
44.343 + def __setitem__(self, key, value):
44.344 + self.shelf[key] = time.time(), value
44.345 +
44.346 + def __delitem__(self, key):
44.347 + try:
44.348 + del self.shelf[key]
44.349 + except KeyError:
44.350 + pass
44.351 +
44.352 + def cleanup(self, timeout):
44.353 + now = time.time()
44.354 + for k in self.shelf.keys():
44.355 + atime, v = self.shelf[k]
44.356 + if now - atime > timeout :
44.357 + del self[k]
44.358 +
44.359 +if __name__ == '__main__' :
44.360 + import doctest
44.361 + doctest.testmod()
45.1 --- /dev/null Thu Jan 01 00:00:00 1970 +0000
45.2 +++ b/OpenSecurity/install/web.py-0.37/build/lib/web/template.py Mon Dec 02 14:02:05 2013 +0100
45.3 @@ -0,0 +1,1515 @@
45.4 +"""
45.5 +simple, elegant templating
45.6 +(part of web.py)
45.7 +
45.8 +Template design:
45.9 +
45.10 +Template string is split into tokens and the tokens are combined into nodes.
45.11 +Parse tree is a nodelist. TextNode and ExpressionNode are simple nodes and
45.12 +for-loop, if-loop etc are block nodes, which contain multiple child nodes.
45.13 +
45.14 +Each node can emit some python string. python string emitted by the
45.15 +root node is validated for safeeval and executed using python in the given environment.
45.16 +
45.17 +Enough care is taken to make sure the generated code and the template has line to line match,
45.18 +so that the error messages can point to exact line number in template. (It doesn't work in some cases still.)
45.19 +
45.20 +Grammar:
45.21 +
45.22 + template -> defwith sections
45.23 + defwith -> '$def with (' arguments ')' | ''
45.24 + sections -> section*
45.25 + section -> block | assignment | line
45.26 +
45.27 + assignment -> '$ ' <assignment expression>
45.28 + line -> (text|expr)*
45.29 + text -> <any characters other than $>
45.30 + expr -> '$' pyexpr | '$(' pyexpr ')' | '${' pyexpr '}'
45.31 + pyexpr -> <python expression>
45.32 +"""
45.33 +
45.34 +__all__ = [
45.35 + "Template",
45.36 + "Render", "render", "frender",
45.37 + "ParseError", "SecurityError",
45.38 + "test"
45.39 +]
45.40 +
45.41 +import tokenize
45.42 +import os
45.43 +import sys
45.44 +import glob
45.45 +import re
45.46 +from UserDict import DictMixin
45.47 +import warnings
45.48 +
45.49 +from utils import storage, safeunicode, safestr, re_compile
45.50 +from webapi import config
45.51 +from net import websafe
45.52 +
45.53 +def splitline(text):
45.54 + r"""
45.55 + Splits the given text at newline.
45.56 +
45.57 + >>> splitline('foo\nbar')
45.58 + ('foo\n', 'bar')
45.59 + >>> splitline('foo')
45.60 + ('foo', '')
45.61 + >>> splitline('')
45.62 + ('', '')
45.63 + """
45.64 + index = text.find('\n') + 1
45.65 + if index:
45.66 + return text[:index], text[index:]
45.67 + else:
45.68 + return text, ''
45.69 +
45.70 +class Parser:
45.71 + """Parser Base.
45.72 + """
45.73 + def __init__(self):
45.74 + self.statement_nodes = STATEMENT_NODES
45.75 + self.keywords = KEYWORDS
45.76 +
45.77 + def parse(self, text, name="<template>"):
45.78 + self.text = text
45.79 + self.name = name
45.80 +
45.81 + defwith, text = self.read_defwith(text)
45.82 + suite = self.read_suite(text)
45.83 + return DefwithNode(defwith, suite)
45.84 +
45.85 + def read_defwith(self, text):
45.86 + if text.startswith('$def with'):
45.87 + defwith, text = splitline(text)
45.88 + defwith = defwith[1:].strip() # strip $ and spaces
45.89 + return defwith, text
45.90 + else:
45.91 + return '', text
45.92 +
45.93 + def read_section(self, text):
45.94 + r"""Reads one section from the given text.
45.95 +
45.96 + section -> block | assignment | line
45.97 +
45.98 + >>> read_section = Parser().read_section
45.99 + >>> read_section('foo\nbar\n')
45.100 + (<line: [t'foo\n']>, 'bar\n')
45.101 + >>> read_section('$ a = b + 1\nfoo\n')
45.102 + (<assignment: 'a = b + 1'>, 'foo\n')
45.103 +
45.104 + read_section('$for in range(10):\n hello $i\nfoo)
45.105 + """
45.106 + if text.lstrip(' ').startswith('$'):
45.107 + index = text.index('$')
45.108 + begin_indent, text2 = text[:index], text[index+1:]
45.109 + ahead = self.python_lookahead(text2)
45.110 +
45.111 + if ahead == 'var':
45.112 + return self.read_var(text2)
45.113 + elif ahead in self.statement_nodes:
45.114 + return self.read_block_section(text2, begin_indent)
45.115 + elif ahead in self.keywords:
45.116 + return self.read_keyword(text2)
45.117 + elif ahead.strip() == '':
45.118 + # assignments starts with a space after $
45.119 + # ex: $ a = b + 2
45.120 + return self.read_assignment(text2)
45.121 + return self.readline(text)
45.122 +
45.123 + def read_var(self, text):
45.124 + r"""Reads a var statement.
45.125 +
45.126 + >>> read_var = Parser().read_var
45.127 + >>> read_var('var x=10\nfoo')
45.128 + (<var: x = 10>, 'foo')
45.129 + >>> read_var('var x: hello $name\nfoo')
45.130 + (<var: x = join_(u'hello ', escape_(name, True))>, 'foo')
45.131 + """
45.132 + line, text = splitline(text)
45.133 + tokens = self.python_tokens(line)
45.134 + if len(tokens) < 4:
45.135 + raise SyntaxError('Invalid var statement')
45.136 +
45.137 + name = tokens[1]
45.138 + sep = tokens[2]
45.139 + value = line.split(sep, 1)[1].strip()
45.140 +
45.141 + if sep == '=':
45.142 + pass # no need to process value
45.143 + elif sep == ':':
45.144 + #@@ Hack for backward-compatability
45.145 + if tokens[3] == '\n': # multi-line var statement
45.146 + block, text = self.read_indented_block(text, ' ')
45.147 + lines = [self.readline(x)[0] for x in block.splitlines()]
45.148 + nodes = []
45.149 + for x in lines:
45.150 + nodes.extend(x.nodes)
45.151 + nodes.append(TextNode('\n'))
45.152 + else: # single-line var statement
45.153 + linenode, _ = self.readline(value)
45.154 + nodes = linenode.nodes
45.155 + parts = [node.emit('') for node in nodes]
45.156 + value = "join_(%s)" % ", ".join(parts)
45.157 + else:
45.158 + raise SyntaxError('Invalid var statement')
45.159 + return VarNode(name, value), text
45.160 +
45.161 + def read_suite(self, text):
45.162 + r"""Reads section by section till end of text.
45.163 +
45.164 + >>> read_suite = Parser().read_suite
45.165 + >>> read_suite('hello $name\nfoo\n')
45.166 + [<line: [t'hello ', $name, t'\n']>, <line: [t'foo\n']>]
45.167 + """
45.168 + sections = []
45.169 + while text:
45.170 + section, text = self.read_section(text)
45.171 + sections.append(section)
45.172 + return SuiteNode(sections)
45.173 +
45.174 + def readline(self, text):
45.175 + r"""Reads one line from the text. Newline is supressed if the line ends with \.
45.176 +
45.177 + >>> readline = Parser().readline
45.178 + >>> readline('hello $name!\nbye!')
45.179 + (<line: [t'hello ', $name, t'!\n']>, 'bye!')
45.180 + >>> readline('hello $name!\\\nbye!')
45.181 + (<line: [t'hello ', $name, t'!']>, 'bye!')
45.182 + >>> readline('$f()\n\n')
45.183 + (<line: [$f(), t'\n']>, '\n')
45.184 + """
45.185 + line, text = splitline(text)
45.186 +
45.187 + # supress new line if line ends with \
45.188 + if line.endswith('\\\n'):
45.189 + line = line[:-2]
45.190 +
45.191 + nodes = []
45.192 + while line:
45.193 + node, line = self.read_node(line)
45.194 + nodes.append(node)
45.195 +
45.196 + return LineNode(nodes), text
45.197 +
45.198 + def read_node(self, text):
45.199 + r"""Reads a node from the given text and returns the node and remaining text.
45.200 +
45.201 + >>> read_node = Parser().read_node
45.202 + >>> read_node('hello $name')
45.203 + (t'hello ', '$name')
45.204 + >>> read_node('$name')
45.205 + ($name, '')
45.206 + """
45.207 + if text.startswith('$$'):
45.208 + return TextNode('$'), text[2:]
45.209 + elif text.startswith('$#'): # comment
45.210 + line, text = splitline(text)
45.211 + return TextNode('\n'), text
45.212 + elif text.startswith('$'):
45.213 + text = text[1:] # strip $
45.214 + if text.startswith(':'):
45.215 + escape = False
45.216 + text = text[1:] # strip :
45.217 + else:
45.218 + escape = True
45.219 + return self.read_expr(text, escape=escape)
45.220 + else:
45.221 + return self.read_text(text)
45.222 +
45.223 + def read_text(self, text):
45.224 + r"""Reads a text node from the given text.
45.225 +
45.226 + >>> read_text = Parser().read_text
45.227 + >>> read_text('hello $name')
45.228 + (t'hello ', '$name')
45.229 + """
45.230 + index = text.find('$')
45.231 + if index < 0:
45.232 + return TextNode(text), ''
45.233 + else:
45.234 + return TextNode(text[:index]), text[index:]
45.235 +
45.236 + def read_keyword(self, text):
45.237 + line, text = splitline(text)
45.238 + return StatementNode(line.strip() + "\n"), text
45.239 +
45.240 + def read_expr(self, text, escape=True):
45.241 + """Reads a python expression from the text and returns the expression and remaining text.
45.242 +
45.243 + expr -> simple_expr | paren_expr
45.244 + simple_expr -> id extended_expr
45.245 + extended_expr -> attr_access | paren_expr extended_expr | ''
45.246 + attr_access -> dot id extended_expr
45.247 + paren_expr -> [ tokens ] | ( tokens ) | { tokens }
45.248 +
45.249 + >>> read_expr = Parser().read_expr
45.250 + >>> read_expr("name")
45.251 + ($name, '')
45.252 + >>> read_expr("a.b and c")
45.253 + ($a.b, ' and c')
45.254 + >>> read_expr("a. b")
45.255 + ($a, '. b')
45.256 + >>> read_expr("name</h1>")
45.257 + ($name, '</h1>')
45.258 + >>> read_expr("(limit)ing")
45.259 + ($(limit), 'ing')
45.260 + >>> read_expr('a[1, 2][:3].f(1+2, "weird string[).", 3 + 4) done.')
45.261 + ($a[1, 2][:3].f(1+2, "weird string[).", 3 + 4), ' done.')
45.262 + """
45.263 + def simple_expr():
45.264 + identifier()
45.265 + extended_expr()
45.266 +
45.267 + def identifier():
45.268 + tokens.next()
45.269 +
45.270 + def extended_expr():
45.271 + lookahead = tokens.lookahead()
45.272 + if lookahead is None:
45.273 + return
45.274 + elif lookahead.value == '.':
45.275 + attr_access()
45.276 + elif lookahead.value in parens:
45.277 + paren_expr()
45.278 + extended_expr()
45.279 + else:
45.280 + return
45.281 +
45.282 + def attr_access():
45.283 + from token import NAME # python token constants
45.284 + dot = tokens.lookahead()
45.285 + if tokens.lookahead2().type == NAME:
45.286 + tokens.next() # consume dot
45.287 + identifier()
45.288 + extended_expr()
45.289 +
45.290 + def paren_expr():
45.291 + begin = tokens.next().value
45.292 + end = parens[begin]
45.293 + while True:
45.294 + if tokens.lookahead().value in parens:
45.295 + paren_expr()
45.296 + else:
45.297 + t = tokens.next()
45.298 + if t.value == end:
45.299 + break
45.300 + return
45.301 +
45.302 + parens = {
45.303 + "(": ")",
45.304 + "[": "]",
45.305 + "{": "}"
45.306 + }
45.307 +
45.308 + def get_tokens(text):
45.309 + """tokenize text using python tokenizer.
45.310 + Python tokenizer ignores spaces, but they might be important in some cases.
45.311 + This function introduces dummy space tokens when it identifies any ignored space.
45.312 + Each token is a storage object containing type, value, begin and end.
45.313 + """
45.314 + readline = iter([text]).next
45.315 + end = None
45.316 + for t in tokenize.generate_tokens(readline):
45.317 + t = storage(type=t[0], value=t[1], begin=t[2], end=t[3])
45.318 + if end is not None and end != t.begin:
45.319 + _, x1 = end
45.320 + _, x2 = t.begin
45.321 + yield storage(type=-1, value=text[x1:x2], begin=end, end=t.begin)
45.322 + end = t.end
45.323 + yield t
45.324 +
45.325 + class BetterIter:
45.326 + """Iterator like object with 2 support for 2 look aheads."""
45.327 + def __init__(self, items):
45.328 + self.iteritems = iter(items)
45.329 + self.items = []
45.330 + self.position = 0
45.331 + self.current_item = None
45.332 +
45.333 + def lookahead(self):
45.334 + if len(self.items) <= self.position:
45.335 + self.items.append(self._next())
45.336 + return self.items[self.position]
45.337 +
45.338 + def _next(self):
45.339 + try:
45.340 + return self.iteritems.next()
45.341 + except StopIteration:
45.342 + return None
45.343 +
45.344 + def lookahead2(self):
45.345 + if len(self.items) <= self.position+1:
45.346 + self.items.append(self._next())
45.347 + return self.items[self.position+1]
45.348 +
45.349 + def next(self):
45.350 + self.current_item = self.lookahead()
45.351 + self.position += 1
45.352 + return self.current_item
45.353 +
45.354 + tokens = BetterIter(get_tokens(text))
45.355 +
45.356 + if tokens.lookahead().value in parens:
45.357 + paren_expr()
45.358 + else:
45.359 + simple_expr()
45.360 + row, col = tokens.current_item.end
45.361 + return ExpressionNode(text[:col], escape=escape), text[col:]
45.362 +
45.363 + def read_assignment(self, text):
45.364 + r"""Reads assignment statement from text.
45.365 +
45.366 + >>> read_assignment = Parser().read_assignment
45.367 + >>> read_assignment('a = b + 1\nfoo')
45.368 + (<assignment: 'a = b + 1'>, 'foo')
45.369 + """
45.370 + line, text = splitline(text)
45.371 + return AssignmentNode(line.strip()), text
45.372 +
45.373 + def python_lookahead(self, text):
45.374 + """Returns the first python token from the given text.
45.375 +
45.376 + >>> python_lookahead = Parser().python_lookahead
45.377 + >>> python_lookahead('for i in range(10):')
45.378 + 'for'
45.379 + >>> python_lookahead('else:')
45.380 + 'else'
45.381 + >>> python_lookahead(' x = 1')
45.382 + ' '
45.383 + """
45.384 + readline = iter([text]).next
45.385 + tokens = tokenize.generate_tokens(readline)
45.386 + return tokens.next()[1]
45.387 +
45.388 + def python_tokens(self, text):
45.389 + readline = iter([text]).next
45.390 + tokens = tokenize.generate_tokens(readline)
45.391 + return [t[1] for t in tokens]
45.392 +
45.393 + def read_indented_block(self, text, indent):
45.394 + r"""Read a block of text. A block is what typically follows a for or it statement.
45.395 + It can be in the same line as that of the statement or an indented block.
45.396 +
45.397 + >>> read_indented_block = Parser().read_indented_block
45.398 + >>> read_indented_block(' a\n b\nc', ' ')
45.399 + ('a\nb\n', 'c')
45.400 + >>> read_indented_block(' a\n b\n c\nd', ' ')
45.401 + ('a\n b\nc\n', 'd')
45.402 + >>> read_indented_block(' a\n\n b\nc', ' ')
45.403 + ('a\n\n b\n', 'c')
45.404 + """
45.405 + if indent == '':
45.406 + return '', text
45.407 +
45.408 + block = ""
45.409 + while text:
45.410 + line, text2 = splitline(text)
45.411 + if line.strip() == "":
45.412 + block += '\n'
45.413 + elif line.startswith(indent):
45.414 + block += line[len(indent):]
45.415 + else:
45.416 + break
45.417 + text = text2
45.418 + return block, text
45.419 +
45.420 + def read_statement(self, text):
45.421 + r"""Reads a python statement.
45.422 +
45.423 + >>> read_statement = Parser().read_statement
45.424 + >>> read_statement('for i in range(10): hello $name')
45.425 + ('for i in range(10):', ' hello $name')
45.426 + """
45.427 + tok = PythonTokenizer(text)
45.428 + tok.consume_till(':')
45.429 + return text[:tok.index], text[tok.index:]
45.430 +
45.431 + def read_block_section(self, text, begin_indent=''):
45.432 + r"""
45.433 + >>> read_block_section = Parser().read_block_section
45.434 + >>> read_block_section('for i in range(10): hello $i\nfoo')
45.435 + (<block: 'for i in range(10):', [<line: [t'hello ', $i, t'\n']>]>, 'foo')
45.436 + >>> read_block_section('for i in range(10):\n hello $i\n foo', begin_indent=' ')
45.437 + (<block: 'for i in range(10):', [<line: [t'hello ', $i, t'\n']>]>, ' foo')
45.438 + >>> read_block_section('for i in range(10):\n hello $i\nfoo')
45.439 + (<block: 'for i in range(10):', [<line: [t'hello ', $i, t'\n']>]>, 'foo')
45.440 + """
45.441 + line, text = splitline(text)
45.442 + stmt, line = self.read_statement(line)
45.443 + keyword = self.python_lookahead(stmt)
45.444 +
45.445 + # if there is some thing left in the line
45.446 + if line.strip():
45.447 + block = line.lstrip()
45.448 + else:
45.449 + def find_indent(text):
45.450 + rx = re_compile(' +')
45.451 + match = rx.match(text)
45.452 + first_indent = match and match.group(0)
45.453 + return first_indent or ""
45.454 +
45.455 + # find the indentation of the block by looking at the first line
45.456 + first_indent = find_indent(text)[len(begin_indent):]
45.457 +
45.458 + #TODO: fix this special case
45.459 + if keyword == "code":
45.460 + indent = begin_indent + first_indent
45.461 + else:
45.462 + indent = begin_indent + min(first_indent, INDENT)
45.463 +
45.464 + block, text = self.read_indented_block(text, indent)
45.465 +
45.466 + return self.create_block_node(keyword, stmt, block, begin_indent), text
45.467 +
45.468 + def create_block_node(self, keyword, stmt, block, begin_indent):
45.469 + if keyword in self.statement_nodes:
45.470 + return self.statement_nodes[keyword](stmt, block, begin_indent)
45.471 + else:
45.472 + raise ParseError, 'Unknown statement: %s' % repr(keyword)
45.473 +
45.474 +class PythonTokenizer:
45.475 + """Utility wrapper over python tokenizer."""
45.476 + def __init__(self, text):
45.477 + self.text = text
45.478 + readline = iter([text]).next
45.479 + self.tokens = tokenize.generate_tokens(readline)
45.480 + self.index = 0
45.481 +
45.482 + def consume_till(self, delim):
45.483 + """Consumes tokens till colon.
45.484 +
45.485 + >>> tok = PythonTokenizer('for i in range(10): hello $i')
45.486 + >>> tok.consume_till(':')
45.487 + >>> tok.text[:tok.index]
45.488 + 'for i in range(10):'
45.489 + >>> tok.text[tok.index:]
45.490 + ' hello $i'
45.491 + """
45.492 + try:
45.493 + while True:
45.494 + t = self.next()
45.495 + if t.value == delim:
45.496 + break
45.497 + elif t.value == '(':
45.498 + self.consume_till(')')
45.499 + elif t.value == '[':
45.500 + self.consume_till(']')
45.501 + elif t.value == '{':
45.502 + self.consume_till('}')
45.503 +
45.504 + # if end of line is found, it is an exception.
45.505 + # Since there is no easy way to report the line number,
45.506 + # leave the error reporting to the python parser later
45.507 + #@@ This should be fixed.
45.508 + if t.value == '\n':
45.509 + break
45.510 + except:
45.511 + #raise ParseError, "Expected %s, found end of line." % repr(delim)
45.512 +
45.513 + # raising ParseError doesn't show the line number.
45.514 + # if this error is ignored, then it will be caught when compiling the python code.
45.515 + return
45.516 +
45.517 + def next(self):
45.518 + type, t, begin, end, line = self.tokens.next()
45.519 + row, col = end
45.520 + self.index = col
45.521 + return storage(type=type, value=t, begin=begin, end=end)
45.522 +
45.523 +class DefwithNode:
45.524 + def __init__(self, defwith, suite):
45.525 + if defwith:
45.526 + self.defwith = defwith.replace('with', '__template__') + ':'
45.527 + # offset 4 lines. for encoding, __lineoffset__, loop and self.
45.528 + self.defwith += "\n __lineoffset__ = -4"
45.529 + else:
45.530 + self.defwith = 'def __template__():'
45.531 + # offset 4 lines for encoding, __template__, __lineoffset__, loop and self.
45.532 + self.defwith += "\n __lineoffset__ = -5"
45.533 +
45.534 + self.defwith += "\n loop = ForLoop()"
45.535 + self.defwith += "\n self = TemplateResult(); extend_ = self.extend"
45.536 + self.suite = suite
45.537 + self.end = "\n return self"
45.538 +
45.539 + def emit(self, indent):
45.540 + encoding = "# coding: utf-8\n"
45.541 + return encoding + self.defwith + self.suite.emit(indent + INDENT) + self.end
45.542 +
45.543 + def __repr__(self):
45.544 + return "<defwith: %s, %s>" % (self.defwith, self.suite)
45.545 +
45.546 +class TextNode:
45.547 + def __init__(self, value):
45.548 + self.value = value
45.549 +
45.550 + def emit(self, indent, begin_indent=''):
45.551 + return repr(safeunicode(self.value))
45.552 +
45.553 + def __repr__(self):
45.554 + return 't' + repr(self.value)
45.555 +
45.556 +class ExpressionNode:
45.557 + def __init__(self, value, escape=True):
45.558 + self.value = value.strip()
45.559 +
45.560 + # convert ${...} to $(...)
45.561 + if value.startswith('{') and value.endswith('}'):
45.562 + self.value = '(' + self.value[1:-1] + ')'
45.563 +
45.564 + self.escape = escape
45.565 +
45.566 + def emit(self, indent, begin_indent=''):
45.567 + return 'escape_(%s, %s)' % (self.value, bool(self.escape))
45.568 +
45.569 + def __repr__(self):
45.570 + if self.escape:
45.571 + escape = ''
45.572 + else:
45.573 + escape = ':'
45.574 + return "$%s%s" % (escape, self.value)
45.575 +
45.576 +class AssignmentNode:
45.577 + def __init__(self, code):
45.578 + self.code = code
45.579 +
45.580 + def emit(self, indent, begin_indent=''):
45.581 + return indent + self.code + "\n"
45.582 +
45.583 + def __repr__(self):
45.584 + return "<assignment: %s>" % repr(self.code)
45.585 +
45.586 +class LineNode:
45.587 + def __init__(self, nodes):
45.588 + self.nodes = nodes
45.589 +
45.590 + def emit(self, indent, text_indent='', name=''):
45.591 + text = [node.emit('') for node in self.nodes]
45.592 + if text_indent:
45.593 + text = [repr(text_indent)] + text
45.594 +
45.595 + return indent + "extend_([%s])\n" % ", ".join(text)
45.596 +
45.597 + def __repr__(self):
45.598 + return "<line: %s>" % repr(self.nodes)
45.599 +
45.600 +INDENT = ' ' # 4 spaces
45.601 +
45.602 +class BlockNode:
45.603 + def __init__(self, stmt, block, begin_indent=''):
45.604 + self.stmt = stmt
45.605 + self.suite = Parser().read_suite(block)
45.606 + self.begin_indent = begin_indent
45.607 +
45.608 + def emit(self, indent, text_indent=''):
45.609 + text_indent = self.begin_indent + text_indent
45.610 + out = indent + self.stmt + self.suite.emit(indent + INDENT, text_indent)
45.611 + return out
45.612 +
45.613 + def __repr__(self):
45.614 + return "<block: %s, %s>" % (repr(self.stmt), repr(self.suite))
45.615 +
45.616 +class ForNode(BlockNode):
45.617 + def __init__(self, stmt, block, begin_indent=''):
45.618 + self.original_stmt = stmt
45.619 + tok = PythonTokenizer(stmt)
45.620 + tok.consume_till('in')
45.621 + a = stmt[:tok.index] # for i in
45.622 + b = stmt[tok.index:-1] # rest of for stmt excluding :
45.623 + stmt = a + ' loop.setup(' + b.strip() + '):'
45.624 + BlockNode.__init__(self, stmt, block, begin_indent)
45.625 +
45.626 + def __repr__(self):
45.627 + return "<block: %s, %s>" % (repr(self.original_stmt), repr(self.suite))
45.628 +
45.629 +class CodeNode:
45.630 + def __init__(self, stmt, block, begin_indent=''):
45.631 + # compensate one line for $code:
45.632 + self.code = "\n" + block
45.633 +
45.634 + def emit(self, indent, text_indent=''):
45.635 + import re
45.636 + rx = re.compile('^', re.M)
45.637 + return rx.sub(indent, self.code).rstrip(' ')
45.638 +
45.639 + def __repr__(self):
45.640 + return "<code: %s>" % repr(self.code)
45.641 +
45.642 +class StatementNode:
45.643 + def __init__(self, stmt):
45.644 + self.stmt = stmt
45.645 +
45.646 + def emit(self, indent, begin_indent=''):
45.647 + return indent + self.stmt
45.648 +
45.649 + def __repr__(self):
45.650 + return "<stmt: %s>" % repr(self.stmt)
45.651 +
45.652 +class IfNode(BlockNode):
45.653 + pass
45.654 +
45.655 +class ElseNode(BlockNode):
45.656 + pass
45.657 +
45.658 +class ElifNode(BlockNode):
45.659 + pass
45.660 +
45.661 +class DefNode(BlockNode):
45.662 + def __init__(self, *a, **kw):
45.663 + BlockNode.__init__(self, *a, **kw)
45.664 +
45.665 + code = CodeNode("", "")
45.666 + code.code = "self = TemplateResult(); extend_ = self.extend\n"
45.667 + self.suite.sections.insert(0, code)
45.668 +
45.669 + code = CodeNode("", "")
45.670 + code.code = "return self\n"
45.671 + self.suite.sections.append(code)
45.672 +
45.673 + def emit(self, indent, text_indent=''):
45.674 + text_indent = self.begin_indent + text_indent
45.675 + out = indent + self.stmt + self.suite.emit(indent + INDENT, text_indent)
45.676 + return indent + "__lineoffset__ -= 3\n" + out
45.677 +
45.678 +class VarNode:
45.679 + def __init__(self, name, value):
45.680 + self.name = name
45.681 + self.value = value
45.682 +
45.683 + def emit(self, indent, text_indent):
45.684 + return indent + "self[%s] = %s\n" % (repr(self.name), self.value)
45.685 +
45.686 + def __repr__(self):
45.687 + return "<var: %s = %s>" % (self.name, self.value)
45.688 +
45.689 +class SuiteNode:
45.690 + """Suite is a list of sections."""
45.691 + def __init__(self, sections):
45.692 + self.sections = sections
45.693 +
45.694 + def emit(self, indent, text_indent=''):
45.695 + return "\n" + "".join([s.emit(indent, text_indent) for s in self.sections])
45.696 +
45.697 + def __repr__(self):
45.698 + return repr(self.sections)
45.699 +
45.700 +STATEMENT_NODES = {
45.701 + 'for': ForNode,
45.702 + 'while': BlockNode,
45.703 + 'if': IfNode,
45.704 + 'elif': ElifNode,
45.705 + 'else': ElseNode,
45.706 + 'def': DefNode,
45.707 + 'code': CodeNode
45.708 +}
45.709 +
45.710 +KEYWORDS = [
45.711 + "pass",
45.712 + "break",
45.713 + "continue",
45.714 + "return"
45.715 +]
45.716 +
45.717 +TEMPLATE_BUILTIN_NAMES = [
45.718 + "dict", "enumerate", "float", "int", "bool", "list", "long", "reversed",
45.719 + "set", "slice", "tuple", "xrange",
45.720 + "abs", "all", "any", "callable", "chr", "cmp", "divmod", "filter", "hex",
45.721 + "id", "isinstance", "iter", "len", "max", "min", "oct", "ord", "pow", "range",
45.722 + "True", "False",
45.723 + "None",
45.724 + "__import__", # some c-libraries like datetime requires __import__ to present in the namespace
45.725 +]
45.726 +
45.727 +import __builtin__
45.728 +TEMPLATE_BUILTINS = dict([(name, getattr(__builtin__, name)) for name in TEMPLATE_BUILTIN_NAMES if name in __builtin__.__dict__])
45.729 +
45.730 +class ForLoop:
45.731 + """
45.732 + Wrapper for expression in for stament to support loop.xxx helpers.
45.733 +
45.734 + >>> loop = ForLoop()
45.735 + >>> for x in loop.setup(['a', 'b', 'c']):
45.736 + ... print loop.index, loop.revindex, loop.parity, x
45.737 + ...
45.738 + 1 3 odd a
45.739 + 2 2 even b
45.740 + 3 1 odd c
45.741 + >>> loop.index
45.742 + Traceback (most recent call last):
45.743 + ...
45.744 + AttributeError: index
45.745 + """
45.746 + def __init__(self):
45.747 + self._ctx = None
45.748 +
45.749 + def __getattr__(self, name):
45.750 + if self._ctx is None:
45.751 + raise AttributeError, name
45.752 + else:
45.753 + return getattr(self._ctx, name)
45.754 +
45.755 + def setup(self, seq):
45.756 + self._push()
45.757 + return self._ctx.setup(seq)
45.758 +
45.759 + def _push(self):
45.760 + self._ctx = ForLoopContext(self, self._ctx)
45.761 +
45.762 + def _pop(self):
45.763 + self._ctx = self._ctx.parent
45.764 +
45.765 +class ForLoopContext:
45.766 + """Stackable context for ForLoop to support nested for loops.
45.767 + """
45.768 + def __init__(self, forloop, parent):
45.769 + self._forloop = forloop
45.770 + self.parent = parent
45.771 +
45.772 + def setup(self, seq):
45.773 + try:
45.774 + self.length = len(seq)
45.775 + except:
45.776 + self.length = 0
45.777 +
45.778 + self.index = 0
45.779 + for a in seq:
45.780 + self.index += 1
45.781 + yield a
45.782 + self._forloop._pop()
45.783 +
45.784 + index0 = property(lambda self: self.index-1)
45.785 + first = property(lambda self: self.index == 1)
45.786 + last = property(lambda self: self.index == self.length)
45.787 + odd = property(lambda self: self.index % 2 == 1)
45.788 + even = property(lambda self: self.index % 2 == 0)
45.789 + parity = property(lambda self: ['odd', 'even'][self.even])
45.790 + revindex0 = property(lambda self: self.length - self.index)
45.791 + revindex = property(lambda self: self.length - self.index + 1)
45.792 +
45.793 +class BaseTemplate:
45.794 + def __init__(self, code, filename, filter, globals, builtins):
45.795 + self.filename = filename
45.796 + self.filter = filter
45.797 + self._globals = globals
45.798 + self._builtins = builtins
45.799 + if code:
45.800 + self.t = self._compile(code)
45.801 + else:
45.802 + self.t = lambda: ''
45.803 +
45.804 + def _compile(self, code):
45.805 + env = self.make_env(self._globals or {}, self._builtins)
45.806 + exec(code, env)
45.807 + return env['__template__']
45.808 +
45.809 + def __call__(self, *a, **kw):
45.810 + __hidetraceback__ = True
45.811 + return self.t(*a, **kw)
45.812 +
45.813 + def make_env(self, globals, builtins):
45.814 + return dict(globals,
45.815 + __builtins__=builtins,
45.816 + ForLoop=ForLoop,
45.817 + TemplateResult=TemplateResult,
45.818 + escape_=self._escape,
45.819 + join_=self._join
45.820 + )
45.821 + def _join(self, *items):
45.822 + return u"".join(items)
45.823 +
45.824 + def _escape(self, value, escape=False):
45.825 + if value is None:
45.826 + value = ''
45.827 +
45.828 + value = safeunicode(value)
45.829 + if escape and self.filter:
45.830 + value = self.filter(value)
45.831 + return value
45.832 +
45.833 +class Template(BaseTemplate):
45.834 + CONTENT_TYPES = {
45.835 + '.html' : 'text/html; charset=utf-8',
45.836 + '.xhtml' : 'application/xhtml+xml; charset=utf-8',
45.837 + '.txt' : 'text/plain',
45.838 + }
45.839 + FILTERS = {
45.840 + '.html': websafe,
45.841 + '.xhtml': websafe,
45.842 + '.xml': websafe
45.843 + }
45.844 + globals = {}
45.845 +
45.846 + def __init__(self, text, filename='<template>', filter=None, globals=None, builtins=None, extensions=None):
45.847 + self.extensions = extensions or []
45.848 + text = Template.normalize_text(text)
45.849 + code = self.compile_template(text, filename)
45.850 +
45.851 + _, ext = os.path.splitext(filename)
45.852 + filter = filter or self.FILTERS.get(ext, None)
45.853 + self.content_type = self.CONTENT_TYPES.get(ext, None)
45.854 +
45.855 + if globals is None:
45.856 + globals = self.globals
45.857 + if builtins is None:
45.858 + builtins = TEMPLATE_BUILTINS
45.859 +
45.860 + BaseTemplate.__init__(self, code=code, filename=filename, filter=filter, globals=globals, builtins=builtins)
45.861 +
45.862 + def normalize_text(text):
45.863 + """Normalizes template text by correcting \r\n, tabs and BOM chars."""
45.864 + text = text.replace('\r\n', '\n').replace('\r', '\n').expandtabs()
45.865 + if not text.endswith('\n'):
45.866 + text += '\n'
45.867 +
45.868 + # ignore BOM chars at the begining of template
45.869 + BOM = '\xef\xbb\xbf'
45.870 + if isinstance(text, str) and text.startswith(BOM):
45.871 + text = text[len(BOM):]
45.872 +
45.873 + # support fort \$ for backward-compatibility
45.874 + text = text.replace(r'\$', '$$')
45.875 + return text
45.876 + normalize_text = staticmethod(normalize_text)
45.877 +
45.878 + def __call__(self, *a, **kw):
45.879 + __hidetraceback__ = True
45.880 + import webapi as web
45.881 + if 'headers' in web.ctx and self.content_type:
45.882 + web.header('Content-Type', self.content_type, unique=True)
45.883 +
45.884 + return BaseTemplate.__call__(self, *a, **kw)
45.885 +
45.886 + def generate_code(text, filename, parser=None):
45.887 + # parse the text
45.888 + parser = parser or Parser()
45.889 + rootnode = parser.parse(text, filename)
45.890 +
45.891 + # generate python code from the parse tree
45.892 + code = rootnode.emit(indent="").strip()
45.893 + return safestr(code)
45.894 +
45.895 + generate_code = staticmethod(generate_code)
45.896 +
45.897 + def create_parser(self):
45.898 + p = Parser()
45.899 + for ext in self.extensions:
45.900 + p = ext(p)
45.901 + return p
45.902 +
45.903 + def compile_template(self, template_string, filename):
45.904 + code = Template.generate_code(template_string, filename, parser=self.create_parser())
45.905 +
45.906 + def get_source_line(filename, lineno):
45.907 + try:
45.908 + lines = open(filename).read().splitlines()
45.909 + return lines[lineno]
45.910 + except:
45.911 + return None
45.912 +
45.913 + try:
45.914 + # compile the code first to report the errors, if any, with the filename
45.915 + compiled_code = compile(code, filename, 'exec')
45.916 + except SyntaxError, e:
45.917 + # display template line that caused the error along with the traceback.
45.918 + try:
45.919 + e.msg += '\n\nTemplate traceback:\n File %s, line %s\n %s' % \
45.920 + (repr(e.filename), e.lineno, get_source_line(e.filename, e.lineno-1))
45.921 + except:
45.922 + pass
45.923 + raise
45.924 +
45.925 + # make sure code is safe - but not with jython, it doesn't have a working compiler module
45.926 + if not sys.platform.startswith('java'):
45.927 + try:
45.928 + import compiler
45.929 + ast = compiler.parse(code)
45.930 + SafeVisitor().walk(ast, filename)
45.931 + except ImportError:
45.932 + warnings.warn("Unabled to import compiler module. Unable to check templates for safety.")
45.933 + else:
45.934 + warnings.warn("SECURITY ISSUE: You are using Jython, which does not support checking templates for safety. Your templates can execute arbitrary code.")
45.935 +
45.936 + return compiled_code
45.937 +
45.938 +class CompiledTemplate(Template):
45.939 + def __init__(self, f, filename):
45.940 + Template.__init__(self, '', filename)
45.941 + self.t = f
45.942 +
45.943 + def compile_template(self, *a):
45.944 + return None
45.945 +
45.946 + def _compile(self, *a):
45.947 + return None
45.948 +
45.949 +class Render:
45.950 + """The most preferred way of using templates.
45.951 +
45.952 + render = web.template.render('templates')
45.953 + print render.foo()
45.954 +
45.955 + Optional parameter can be `base` can be used to pass output of
45.956 + every template through the base template.
45.957 +
45.958 + render = web.template.render('templates', base='layout')
45.959 + """
45.960 + def __init__(self, loc='templates', cache=None, base=None, **keywords):
45.961 + self._loc = loc
45.962 + self._keywords = keywords
45.963 +
45.964 + if cache is None:
45.965 + cache = not config.get('debug', False)
45.966 +
45.967 + if cache:
45.968 + self._cache = {}
45.969 + else:
45.970 + self._cache = None
45.971 +
45.972 + if base and not hasattr(base, '__call__'):
45.973 + # make base a function, so that it can be passed to sub-renders
45.974 + self._base = lambda page: self._template(base)(page)
45.975 + else:
45.976 + self._base = base
45.977 +
45.978 + def _add_global(self, obj, name=None):
45.979 + """Add a global to this rendering instance."""
45.980 + if 'globals' not in self._keywords: self._keywords['globals'] = {}
45.981 + if not name:
45.982 + name = obj.__name__
45.983 + self._keywords['globals'][name] = obj
45.984 +
45.985 + def _lookup(self, name):
45.986 + path = os.path.join(self._loc, name)
45.987 + if os.path.isdir(path):
45.988 + return 'dir', path
45.989 + else:
45.990 + path = self._findfile(path)
45.991 + if path:
45.992 + return 'file', path
45.993 + else:
45.994 + return 'none', None
45.995 +
45.996 + def _load_template(self, name):
45.997 + kind, path = self._lookup(name)
45.998 +
45.999 + if kind == 'dir':
45.1000 + return Render(path, cache=self._cache is not None, base=self._base, **self._keywords)
45.1001 + elif kind == 'file':
45.1002 + return Template(open(path).read(), filename=path, **self._keywords)
45.1003 + else:
45.1004 + raise AttributeError, "No template named " + name
45.1005 +
45.1006 + def _findfile(self, path_prefix):
45.1007 + p = [f for f in glob.glob(path_prefix + '.*') if not f.endswith('~')] # skip backup files
45.1008 + p.sort() # sort the matches for deterministic order
45.1009 + return p and p[0]
45.1010 +
45.1011 + def _template(self, name):
45.1012 + if self._cache is not None:
45.1013 + if name not in self._cache:
45.1014 + self._cache[name] = self._load_template(name)
45.1015 + return self._cache[name]
45.1016 + else:
45.1017 + return self._load_template(name)
45.1018 +
45.1019 + def __getattr__(self, name):
45.1020 + t = self._template(name)
45.1021 + if self._base and isinstance(t, Template):
45.1022 + def template(*a, **kw):
45.1023 + return self._base(t(*a, **kw))
45.1024 + return template
45.1025 + else:
45.1026 + return self._template(name)
45.1027 +
45.1028 +class GAE_Render(Render):
45.1029 + # Render gets over-written. make a copy here.
45.1030 + super = Render
45.1031 + def __init__(self, loc, *a, **kw):
45.1032 + GAE_Render.super.__init__(self, loc, *a, **kw)
45.1033 +
45.1034 + import types
45.1035 + if isinstance(loc, types.ModuleType):
45.1036 + self.mod = loc
45.1037 + else:
45.1038 + name = loc.rstrip('/').replace('/', '.')
45.1039 + self.mod = __import__(name, None, None, ['x'])
45.1040 +
45.1041 + self.mod.__dict__.update(kw.get('builtins', TEMPLATE_BUILTINS))
45.1042 + self.mod.__dict__.update(Template.globals)
45.1043 + self.mod.__dict__.update(kw.get('globals', {}))
45.1044 +
45.1045 + def _load_template(self, name):
45.1046 + t = getattr(self.mod, name)
45.1047 + import types
45.1048 + if isinstance(t, types.ModuleType):
45.1049 + return GAE_Render(t, cache=self._cache is not None, base=self._base, **self._keywords)
45.1050 + else:
45.1051 + return t
45.1052 +
45.1053 +render = Render
45.1054 +# setup render for Google App Engine.
45.1055 +try:
45.1056 + from google import appengine
45.1057 + render = Render = GAE_Render
45.1058 +except ImportError:
45.1059 + pass
45.1060 +
45.1061 +def frender(path, **keywords):
45.1062 + """Creates a template from the given file path.
45.1063 + """
45.1064 + return Template(open(path).read(), filename=path, **keywords)
45.1065 +
45.1066 +def compile_templates(root):
45.1067 + """Compiles templates to python code."""
45.1068 + re_start = re_compile('^', re.M)
45.1069 +
45.1070 + for dirpath, dirnames, filenames in os.walk(root):
45.1071 + filenames = [f for f in filenames if not f.startswith('.') and not f.endswith('~') and not f.startswith('__init__.py')]
45.1072 +
45.1073 + for d in dirnames[:]:
45.1074 + if d.startswith('.'):
45.1075 + dirnames.remove(d) # don't visit this dir
45.1076 +
45.1077 + out = open(os.path.join(dirpath, '__init__.py'), 'w')
45.1078 + out.write('from web.template import CompiledTemplate, ForLoop, TemplateResult\n\n')
45.1079 + if dirnames:
45.1080 + out.write("import " + ", ".join(dirnames))
45.1081 + out.write("\n")
45.1082 +
45.1083 + for f in filenames:
45.1084 + path = os.path.join(dirpath, f)
45.1085 +
45.1086 + if '.' in f:
45.1087 + name, _ = f.split('.', 1)
45.1088 + else:
45.1089 + name = f
45.1090 +
45.1091 + text = open(path).read()
45.1092 + text = Template.normalize_text(text)
45.1093 + code = Template.generate_code(text, path)
45.1094 +
45.1095 + code = code.replace("__template__", name, 1)
45.1096 +
45.1097 + out.write(code)
45.1098 +
45.1099 + out.write('\n\n')
45.1100 + out.write('%s = CompiledTemplate(%s, %s)\n' % (name, name, repr(path)))
45.1101 + out.write("join_ = %s._join; escape_ = %s._escape\n\n" % (name, name))
45.1102 +
45.1103 + # create template to make sure it compiles
45.1104 + t = Template(open(path).read(), path)
45.1105 + out.close()
45.1106 +
45.1107 +class ParseError(Exception):
45.1108 + pass
45.1109 +
45.1110 +class SecurityError(Exception):
45.1111 + """The template seems to be trying to do something naughty."""
45.1112 + pass
45.1113 +
45.1114 +# Enumerate all the allowed AST nodes
45.1115 +ALLOWED_AST_NODES = [
45.1116 + "Add", "And",
45.1117 +# "AssAttr",
45.1118 + "AssList", "AssName", "AssTuple",
45.1119 +# "Assert",
45.1120 + "Assign", "AugAssign",
45.1121 +# "Backquote",
45.1122 + "Bitand", "Bitor", "Bitxor", "Break",
45.1123 + "CallFunc","Class", "Compare", "Const", "Continue",
45.1124 + "Decorators", "Dict", "Discard", "Div",
45.1125 + "Ellipsis", "EmptyNode",
45.1126 +# "Exec",
45.1127 + "Expression", "FloorDiv", "For",
45.1128 +# "From",
45.1129 + "Function",
45.1130 + "GenExpr", "GenExprFor", "GenExprIf", "GenExprInner",
45.1131 + "Getattr",
45.1132 +# "Global",
45.1133 + "If", "IfExp",
45.1134 +# "Import",
45.1135 + "Invert", "Keyword", "Lambda", "LeftShift",
45.1136 + "List", "ListComp", "ListCompFor", "ListCompIf", "Mod",
45.1137 + "Module",
45.1138 + "Mul", "Name", "Not", "Or", "Pass", "Power",
45.1139 +# "Print", "Printnl", "Raise",
45.1140 + "Return", "RightShift", "Slice", "Sliceobj",
45.1141 + "Stmt", "Sub", "Subscript",
45.1142 +# "TryExcept", "TryFinally",
45.1143 + "Tuple", "UnaryAdd", "UnarySub",
45.1144 + "While", "With", "Yield",
45.1145 +]
45.1146 +
45.1147 +class SafeVisitor(object):
45.1148 + """
45.1149 + Make sure code is safe by walking through the AST.
45.1150 +
45.1151 + Code considered unsafe if:
45.1152 + * it has restricted AST nodes
45.1153 + * it is trying to access resricted attributes
45.1154 +
45.1155 + Adopted from http://www.zafar.se/bkz/uploads/safe.txt (public domain, Babar K. Zafar)
45.1156 + """
45.1157 + def __init__(self):
45.1158 + "Initialize visitor by generating callbacks for all AST node types."
45.1159 + self.errors = []
45.1160 +
45.1161 + def walk(self, ast, filename):
45.1162 + "Validate each node in AST and raise SecurityError if the code is not safe."
45.1163 + self.filename = filename
45.1164 + self.visit(ast)
45.1165 +
45.1166 + if self.errors:
45.1167 + raise SecurityError, '\n'.join([str(err) for err in self.errors])
45.1168 +
45.1169 + def visit(self, node, *args):
45.1170 + "Recursively validate node and all of its children."
45.1171 + def classname(obj):
45.1172 + return obj.__class__.__name__
45.1173 + nodename = classname(node)
45.1174 + fn = getattr(self, 'visit' + nodename, None)
45.1175 +
45.1176 + if fn:
45.1177 + fn(node, *args)
45.1178 + else:
45.1179 + if nodename not in ALLOWED_AST_NODES:
45.1180 + self.fail(node, *args)
45.1181 +
45.1182 + for child in node.getChildNodes():
45.1183 + self.visit(child, *args)
45.1184 +
45.1185 + def visitName(self, node, *args):
45.1186 + "Disallow any attempts to access a restricted attr."
45.1187 + #self.assert_attr(node.getChildren()[0], node)
45.1188 + pass
45.1189 +
45.1190 + def visitGetattr(self, node, *args):
45.1191 + "Disallow any attempts to access a restricted attribute."
45.1192 + self.assert_attr(node.attrname, node)
45.1193 +
45.1194 + def assert_attr(self, attrname, node):
45.1195 + if self.is_unallowed_attr(attrname):
45.1196 + lineno = self.get_node_lineno(node)
45.1197 + e = SecurityError("%s:%d - access to attribute '%s' is denied" % (self.filename, lineno, attrname))
45.1198 + self.errors.append(e)
45.1199 +
45.1200 + def is_unallowed_attr(self, name):
45.1201 + return name.startswith('_') \
45.1202 + or name.startswith('func_') \
45.1203 + or name.startswith('im_')
45.1204 +
45.1205 + def get_node_lineno(self, node):
45.1206 + return (node.lineno) and node.lineno or 0
45.1207 +
45.1208 + def fail(self, node, *args):
45.1209 + "Default callback for unallowed AST nodes."
45.1210 + lineno = self.get_node_lineno(node)
45.1211 + nodename = node.__class__.__name__
45.1212 + e = SecurityError("%s:%d - execution of '%s' statements is denied" % (self.filename, lineno, nodename))
45.1213 + self.errors.append(e)
45.1214 +
45.1215 +class TemplateResult(object, DictMixin):
45.1216 + """Dictionary like object for storing template output.
45.1217 +
45.1218 + The result of a template execution is usally a string, but sometimes it
45.1219 + contains attributes set using $var. This class provides a simple
45.1220 + dictionary like interface for storing the output of the template and the
45.1221 + attributes. The output is stored with a special key __body__. Convering
45.1222 + the the TemplateResult to string or unicode returns the value of __body__.
45.1223 +
45.1224 + When the template is in execution, the output is generated part by part
45.1225 + and those parts are combined at the end. Parts are added to the
45.1226 + TemplateResult by calling the `extend` method and the parts are combined
45.1227 + seemlessly when __body__ is accessed.
45.1228 +
45.1229 + >>> d = TemplateResult(__body__='hello, world', x='foo')
45.1230 + >>> d
45.1231 + <TemplateResult: {'__body__': 'hello, world', 'x': 'foo'}>
45.1232 + >>> print d
45.1233 + hello, world
45.1234 + >>> d.x
45.1235 + 'foo'
45.1236 + >>> d = TemplateResult()
45.1237 + >>> d.extend([u'hello', u'world'])
45.1238 + >>> d
45.1239 + <TemplateResult: {'__body__': u'helloworld'}>
45.1240 + """
45.1241 + def __init__(self, *a, **kw):
45.1242 + self.__dict__["_d"] = dict(*a, **kw)
45.1243 + self._d.setdefault("__body__", u'')
45.1244 +
45.1245 + self.__dict__['_parts'] = []
45.1246 + self.__dict__["extend"] = self._parts.extend
45.1247 +
45.1248 + self._d.setdefault("__body__", None)
45.1249 +
45.1250 + def keys(self):
45.1251 + return self._d.keys()
45.1252 +
45.1253 + def _prepare_body(self):
45.1254 + """Prepare value of __body__ by joining parts.
45.1255 + """
45.1256 + if self._parts:
45.1257 + value = u"".join(self._parts)
45.1258 + self._parts[:] = []
45.1259 + body = self._d.get('__body__')
45.1260 + if body:
45.1261 + self._d['__body__'] = body + value
45.1262 + else:
45.1263 + self._d['__body__'] = value
45.1264 +
45.1265 + def __getitem__(self, name):
45.1266 + if name == "__body__":
45.1267 + self._prepare_body()
45.1268 + return self._d[name]
45.1269 +
45.1270 + def __setitem__(self, name, value):
45.1271 + if name == "__body__":
45.1272 + self._prepare_body()
45.1273 + return self._d.__setitem__(name, value)
45.1274 +
45.1275 + def __delitem__(self, name):
45.1276 + if name == "__body__":
45.1277 + self._prepare_body()
45.1278 + return self._d.__delitem__(name)
45.1279 +
45.1280 + def __getattr__(self, key):
45.1281 + try:
45.1282 + return self[key]
45.1283 + except KeyError, k:
45.1284 + raise AttributeError, k
45.1285 +
45.1286 + def __setattr__(self, key, value):
45.1287 + self[key] = value
45.1288 +
45.1289 + def __delattr__(self, key):
45.1290 + try:
45.1291 + del self[key]
45.1292 + except KeyError, k:
45.1293 + raise AttributeError, k
45.1294 +
45.1295 + def __unicode__(self):
45.1296 + self._prepare_body()
45.1297 + return self["__body__"]
45.1298 +
45.1299 + def __str__(self):
45.1300 + self._prepare_body()
45.1301 + return self["__body__"].encode('utf-8')
45.1302 +
45.1303 + def __repr__(self):
45.1304 + self._prepare_body()
45.1305 + return "<TemplateResult: %s>" % self._d
45.1306 +
45.1307 +def test():
45.1308 + r"""Doctest for testing template module.
45.1309 +
45.1310 + Define a utility function to run template test.
45.1311 +
45.1312 + >>> class TestResult:
45.1313 + ... def __init__(self, t): self.t = t
45.1314 + ... def __getattr__(self, name): return getattr(self.t, name)
45.1315 + ... def __repr__(self): return repr(unicode(self))
45.1316 + ...
45.1317 + >>> def t(code, **keywords):
45.1318 + ... tmpl = Template(code, **keywords)
45.1319 + ... return lambda *a, **kw: TestResult(tmpl(*a, **kw))
45.1320 + ...
45.1321 +
45.1322 + Simple tests.
45.1323 +
45.1324 + >>> t('1')()
45.1325 + u'1\n'
45.1326 + >>> t('$def with ()\n1')()
45.1327 + u'1\n'
45.1328 + >>> t('$def with (a)\n$a')(1)
45.1329 + u'1\n'
45.1330 + >>> t('$def with (a=0)\n$a')(1)
45.1331 + u'1\n'
45.1332 + >>> t('$def with (a=0)\n$a')(a=1)
45.1333 + u'1\n'
45.1334 +
45.1335 + Test complicated expressions.
45.1336 +
45.1337 + >>> t('$def with (x)\n$x.upper()')('hello')
45.1338 + u'HELLO\n'
45.1339 + >>> t('$(2 * 3 + 4 * 5)')()
45.1340 + u'26\n'
45.1341 + >>> t('${2 * 3 + 4 * 5}')()
45.1342 + u'26\n'
45.1343 + >>> t('$def with (limit)\nkeep $(limit)ing.')('go')
45.1344 + u'keep going.\n'
45.1345 + >>> t('$def with (a)\n$a.b[0]')(storage(b=[1]))
45.1346 + u'1\n'
45.1347 +
45.1348 + Test html escaping.
45.1349 +
45.1350 + >>> t('$def with (x)\n$x', filename='a.html')('<html>')
45.1351 + u'<html>\n'
45.1352 + >>> t('$def with (x)\n$x', filename='a.txt')('<html>')
45.1353 + u'<html>\n'
45.1354 +
45.1355 + Test if, for and while.
45.1356 +
45.1357 + >>> t('$if 1: 1')()
45.1358 + u'1\n'
45.1359 + >>> t('$if 1:\n 1')()
45.1360 + u'1\n'
45.1361 + >>> t('$if 1:\n 1\\')()
45.1362 + u'1'
45.1363 + >>> t('$if 0: 0\n$elif 1: 1')()
45.1364 + u'1\n'
45.1365 + >>> t('$if 0: 0\n$elif None: 0\n$else: 1')()
45.1366 + u'1\n'
45.1367 + >>> t('$if 0 < 1 and 1 < 2: 1')()
45.1368 + u'1\n'
45.1369 + >>> t('$for x in [1, 2, 3]: $x')()
45.1370 + u'1\n2\n3\n'
45.1371 + >>> t('$def with (d)\n$for k, v in d.iteritems(): $k')({1: 1})
45.1372 + u'1\n'
45.1373 + >>> t('$for x in [1, 2, 3]:\n\t$x')()
45.1374 + u' 1\n 2\n 3\n'
45.1375 + >>> t('$def with (a)\n$while a and a.pop():1')([1, 2, 3])
45.1376 + u'1\n1\n1\n'
45.1377 +
45.1378 + The space after : must be ignored.
45.1379 +
45.1380 + >>> t('$if True: foo')()
45.1381 + u'foo\n'
45.1382 +
45.1383 + Test loop.xxx.
45.1384 +
45.1385 + >>> t("$for i in range(5):$loop.index, $loop.parity")()
45.1386 + u'1, odd\n2, even\n3, odd\n4, even\n5, odd\n'
45.1387 + >>> t("$for i in range(2):\n $for j in range(2):$loop.parent.parity $loop.parity")()
45.1388 + u'odd odd\nodd even\neven odd\neven even\n'
45.1389 +
45.1390 + Test assignment.
45.1391 +
45.1392 + >>> t('$ a = 1\n$a')()
45.1393 + u'1\n'
45.1394 + >>> t('$ a = [1]\n$a[0]')()
45.1395 + u'1\n'
45.1396 + >>> t('$ a = {1: 1}\n$a.keys()[0]')()
45.1397 + u'1\n'
45.1398 + >>> t('$ a = []\n$if not a: 1')()
45.1399 + u'1\n'
45.1400 + >>> t('$ a = {}\n$if not a: 1')()
45.1401 + u'1\n'
45.1402 + >>> t('$ a = -1\n$a')()
45.1403 + u'-1\n'
45.1404 + >>> t('$ a = "1"\n$a')()
45.1405 + u'1\n'
45.1406 +
45.1407 + Test comments.
45.1408 +
45.1409 + >>> t('$# 0')()
45.1410 + u'\n'
45.1411 + >>> t('hello$#comment1\nhello$#comment2')()
45.1412 + u'hello\nhello\n'
45.1413 + >>> t('$#comment0\nhello$#comment1\nhello$#comment2')()
45.1414 + u'\nhello\nhello\n'
45.1415 +
45.1416 + Test unicode.
45.1417 +
45.1418 + >>> t('$def with (a)\n$a')(u'\u203d')
45.1419 + u'\u203d\n'
45.1420 + >>> t('$def with (a)\n$a')(u'\u203d'.encode('utf-8'))
45.1421 + u'\u203d\n'
45.1422 + >>> t(u'$def with (a)\n$a $:a')(u'\u203d')
45.1423 + u'\u203d \u203d\n'
45.1424 + >>> t(u'$def with ()\nfoo')()
45.1425 + u'foo\n'
45.1426 + >>> def f(x): return x
45.1427 + ...
45.1428 + >>> t(u'$def with (f)\n$:f("x")')(f)
45.1429 + u'x\n'
45.1430 + >>> t('$def with (f)\n$:f("x")')(f)
45.1431 + u'x\n'
45.1432 +
45.1433 + Test dollar escaping.
45.1434 +
45.1435 + >>> t("Stop, $$money isn't evaluated.")()
45.1436 + u"Stop, $money isn't evaluated.\n"
45.1437 + >>> t("Stop, \$money isn't evaluated.")()
45.1438 + u"Stop, $money isn't evaluated.\n"
45.1439 +
45.1440 + Test space sensitivity.
45.1441 +
45.1442 + >>> t('$def with (x)\n$x')(1)
45.1443 + u'1\n'
45.1444 + >>> t('$def with(x ,y)\n$x')(1, 1)
45.1445 + u'1\n'
45.1446 + >>> t('$(1 + 2*3 + 4)')()
45.1447 + u'11\n'
45.1448 +
45.1449 + Make sure globals are working.
45.1450 +
45.1451 + >>> t('$x')()
45.1452 + Traceback (most recent call last):
45.1453 + ...
45.1454 + NameError: global name 'x' is not defined
45.1455 + >>> t('$x', globals={'x': 1})()
45.1456 + u'1\n'
45.1457 +
45.1458 + Can't change globals.
45.1459 +
45.1460 + >>> t('$ x = 2\n$x', globals={'x': 1})()
45.1461 + u'2\n'
45.1462 + >>> t('$ x = x + 1\n$x', globals={'x': 1})()
45.1463 + Traceback (most recent call last):
45.1464 + ...
45.1465 + UnboundLocalError: local variable 'x' referenced before assignment
45.1466 +
45.1467 + Make sure builtins are customizable.
45.1468 +
45.1469 + >>> t('$min(1, 2)')()
45.1470 + u'1\n'
45.1471 + >>> t('$min(1, 2)', builtins={})()
45.1472 + Traceback (most recent call last):
45.1473 + ...
45.1474 + NameError: global name 'min' is not defined
45.1475 +
45.1476 + Test vars.
45.1477 +
45.1478 + >>> x = t('$var x: 1')()
45.1479 + >>> x.x
45.1480 + u'1'
45.1481 + >>> x = t('$var x = 1')()
45.1482 + >>> x.x
45.1483 + 1
45.1484 + >>> x = t('$var x: \n foo\n bar')()
45.1485 + >>> x.x
45.1486 + u'foo\nbar\n'
45.1487 +
45.1488 + Test BOM chars.
45.1489 +
45.1490 + >>> t('\xef\xbb\xbf$def with(x)\n$x')('foo')
45.1491 + u'foo\n'
45.1492 +
45.1493 + Test for with weird cases.
45.1494 +
45.1495 + >>> t('$for i in range(10)[1:5]:\n $i')()
45.1496 + u'1\n2\n3\n4\n'
45.1497 + >>> t("$for k, v in {'a': 1, 'b': 2}.items():\n $k $v")()
45.1498 + u'a 1\nb 2\n'
45.1499 + >>> t("$for k, v in ({'a': 1, 'b': 2}.items():\n $k $v")()
45.1500 + Traceback (most recent call last):
45.1501 + ...
45.1502 + SyntaxError: invalid syntax
45.1503 +
45.1504 + Test datetime.
45.1505 +
45.1506 + >>> import datetime
45.1507 + >>> t("$def with (date)\n$date.strftime('%m %Y')")(datetime.datetime(2009, 1, 1))
45.1508 + u'01 2009\n'
45.1509 + """
45.1510 + pass
45.1511 +
45.1512 +if __name__ == "__main__":
45.1513 + import sys
45.1514 + if '--compile' in sys.argv:
45.1515 + compile_templates(sys.argv[2])
45.1516 + else:
45.1517 + import doctest
45.1518 + doctest.testmod()
46.1 --- /dev/null Thu Jan 01 00:00:00 1970 +0000
46.2 +++ b/OpenSecurity/install/web.py-0.37/build/lib/web/test.py Mon Dec 02 14:02:05 2013 +0100
46.3 @@ -0,0 +1,51 @@
46.4 +"""test utilities
46.5 +(part of web.py)
46.6 +"""
46.7 +import unittest
46.8 +import sys, os
46.9 +import web
46.10 +
46.11 +TestCase = unittest.TestCase
46.12 +TestSuite = unittest.TestSuite
46.13 +
46.14 +def load_modules(names):
46.15 + return [__import__(name, None, None, "x") for name in names]
46.16 +
46.17 +def module_suite(module, classnames=None):
46.18 + """Makes a suite from a module."""
46.19 + if classnames:
46.20 + return unittest.TestLoader().loadTestsFromNames(classnames, module)
46.21 + elif hasattr(module, 'suite'):
46.22 + return module.suite()
46.23 + else:
46.24 + return unittest.TestLoader().loadTestsFromModule(module)
46.25 +
46.26 +def doctest_suite(module_names):
46.27 + """Makes a test suite from doctests."""
46.28 + import doctest
46.29 + suite = TestSuite()
46.30 + for mod in load_modules(module_names):
46.31 + suite.addTest(doctest.DocTestSuite(mod))
46.32 + return suite
46.33 +
46.34 +def suite(module_names):
46.35 + """Creates a suite from multiple modules."""
46.36 + suite = TestSuite()
46.37 + for mod in load_modules(module_names):
46.38 + suite.addTest(module_suite(mod))
46.39 + return suite
46.40 +
46.41 +def runTests(suite):
46.42 + runner = unittest.TextTestRunner()
46.43 + return runner.run(suite)
46.44 +
46.45 +def main(suite=None):
46.46 + if not suite:
46.47 + main_module = __import__('__main__')
46.48 + # allow command line switches
46.49 + args = [a for a in sys.argv[1:] if not a.startswith('-')]
46.50 + suite = module_suite(main_module, args or None)
46.51 +
46.52 + result = runTests(suite)
46.53 + sys.exit(not result.wasSuccessful())
46.54 +
47.1 --- /dev/null Thu Jan 01 00:00:00 1970 +0000
47.2 +++ b/OpenSecurity/install/web.py-0.37/build/lib/web/utils.py Mon Dec 02 14:02:05 2013 +0100
47.3 @@ -0,0 +1,1526 @@
47.4 +#!/usr/bin/env python
47.5 +"""
47.6 +General Utilities
47.7 +(part of web.py)
47.8 +"""
47.9 +
47.10 +__all__ = [
47.11 + "Storage", "storage", "storify",
47.12 + "Counter", "counter",
47.13 + "iters",
47.14 + "rstrips", "lstrips", "strips",
47.15 + "safeunicode", "safestr", "utf8",
47.16 + "TimeoutError", "timelimit",
47.17 + "Memoize", "memoize",
47.18 + "re_compile", "re_subm",
47.19 + "group", "uniq", "iterview",
47.20 + "IterBetter", "iterbetter",
47.21 + "safeiter", "safewrite",
47.22 + "dictreverse", "dictfind", "dictfindall", "dictincr", "dictadd",
47.23 + "requeue", "restack",
47.24 + "listget", "intget", "datestr",
47.25 + "numify", "denumify", "commify", "dateify",
47.26 + "nthstr", "cond",
47.27 + "CaptureStdout", "capturestdout", "Profile", "profile",
47.28 + "tryall",
47.29 + "ThreadedDict", "threadeddict",
47.30 + "autoassign",
47.31 + "to36",
47.32 + "safemarkdown",
47.33 + "sendmail"
47.34 +]
47.35 +
47.36 +import re, sys, time, threading, itertools, traceback, os
47.37 +
47.38 +try:
47.39 + import subprocess
47.40 +except ImportError:
47.41 + subprocess = None
47.42 +
47.43 +try: import datetime
47.44 +except ImportError: pass
47.45 +
47.46 +try: set
47.47 +except NameError:
47.48 + from sets import Set as set
47.49 +
47.50 +try:
47.51 + from threading import local as threadlocal
47.52 +except ImportError:
47.53 + from python23 import threadlocal
47.54 +
47.55 +class Storage(dict):
47.56 + """
47.57 + A Storage object is like a dictionary except `obj.foo` can be used
47.58 + in addition to `obj['foo']`.
47.59 +
47.60 + >>> o = storage(a=1)
47.61 + >>> o.a
47.62 + 1
47.63 + >>> o['a']
47.64 + 1
47.65 + >>> o.a = 2
47.66 + >>> o['a']
47.67 + 2
47.68 + >>> del o.a
47.69 + >>> o.a
47.70 + Traceback (most recent call last):
47.71 + ...
47.72 + AttributeError: 'a'
47.73 +
47.74 + """
47.75 + def __getattr__(self, key):
47.76 + try:
47.77 + return self[key]
47.78 + except KeyError, k:
47.79 + raise AttributeError, k
47.80 +
47.81 + def __setattr__(self, key, value):
47.82 + self[key] = value
47.83 +
47.84 + def __delattr__(self, key):
47.85 + try:
47.86 + del self[key]
47.87 + except KeyError, k:
47.88 + raise AttributeError, k
47.89 +
47.90 + def __repr__(self):
47.91 + return '<Storage ' + dict.__repr__(self) + '>'
47.92 +
47.93 +storage = Storage
47.94 +
47.95 +def storify(mapping, *requireds, **defaults):
47.96 + """
47.97 + Creates a `storage` object from dictionary `mapping`, raising `KeyError` if
47.98 + d doesn't have all of the keys in `requireds` and using the default
47.99 + values for keys found in `defaults`.
47.100 +
47.101 + For example, `storify({'a':1, 'c':3}, b=2, c=0)` will return the equivalent of
47.102 + `storage({'a':1, 'b':2, 'c':3})`.
47.103 +
47.104 + If a `storify` value is a list (e.g. multiple values in a form submission),
47.105 + `storify` returns the last element of the list, unless the key appears in
47.106 + `defaults` as a list. Thus:
47.107 +
47.108 + >>> storify({'a':[1, 2]}).a
47.109 + 2
47.110 + >>> storify({'a':[1, 2]}, a=[]).a
47.111 + [1, 2]
47.112 + >>> storify({'a':1}, a=[]).a
47.113 + [1]
47.114 + >>> storify({}, a=[]).a
47.115 + []
47.116 +
47.117 + Similarly, if the value has a `value` attribute, `storify will return _its_
47.118 + value, unless the key appears in `defaults` as a dictionary.
47.119 +
47.120 + >>> storify({'a':storage(value=1)}).a
47.121 + 1
47.122 + >>> storify({'a':storage(value=1)}, a={}).a
47.123 + <Storage {'value': 1}>
47.124 + >>> storify({}, a={}).a
47.125 + {}
47.126 +
47.127 + Optionally, keyword parameter `_unicode` can be passed to convert all values to unicode.
47.128 +
47.129 + >>> storify({'x': 'a'}, _unicode=True)
47.130 + <Storage {'x': u'a'}>
47.131 + >>> storify({'x': storage(value='a')}, x={}, _unicode=True)
47.132 + <Storage {'x': <Storage {'value': 'a'}>}>
47.133 + >>> storify({'x': storage(value='a')}, _unicode=True)
47.134 + <Storage {'x': u'a'}>
47.135 + """
47.136 + _unicode = defaults.pop('_unicode', False)
47.137 +
47.138 + # if _unicode is callable object, use it convert a string to unicode.
47.139 + to_unicode = safeunicode
47.140 + if _unicode is not False and hasattr(_unicode, "__call__"):
47.141 + to_unicode = _unicode
47.142 +
47.143 + def unicodify(s):
47.144 + if _unicode and isinstance(s, str): return to_unicode(s)
47.145 + else: return s
47.146 +
47.147 + def getvalue(x):
47.148 + if hasattr(x, 'file') and hasattr(x, 'value'):
47.149 + return x.value
47.150 + elif hasattr(x, 'value'):
47.151 + return unicodify(x.value)
47.152 + else:
47.153 + return unicodify(x)
47.154 +
47.155 + stor = Storage()
47.156 + for key in requireds + tuple(mapping.keys()):
47.157 + value = mapping[key]
47.158 + if isinstance(value, list):
47.159 + if isinstance(defaults.get(key), list):
47.160 + value = [getvalue(x) for x in value]
47.161 + else:
47.162 + value = value[-1]
47.163 + if not isinstance(defaults.get(key), dict):
47.164 + value = getvalue(value)
47.165 + if isinstance(defaults.get(key), list) and not isinstance(value, list):
47.166 + value = [value]
47.167 + setattr(stor, key, value)
47.168 +
47.169 + for (key, value) in defaults.iteritems():
47.170 + result = value
47.171 + if hasattr(stor, key):
47.172 + result = stor[key]
47.173 + if value == () and not isinstance(result, tuple):
47.174 + result = (result,)
47.175 + setattr(stor, key, result)
47.176 +
47.177 + return stor
47.178 +
47.179 +class Counter(storage):
47.180 + """Keeps count of how many times something is added.
47.181 +
47.182 + >>> c = counter()
47.183 + >>> c.add('x')
47.184 + >>> c.add('x')
47.185 + >>> c.add('x')
47.186 + >>> c.add('x')
47.187 + >>> c.add('x')
47.188 + >>> c.add('y')
47.189 + >>> c
47.190 + <Counter {'y': 1, 'x': 5}>
47.191 + >>> c.most()
47.192 + ['x']
47.193 + """
47.194 + def add(self, n):
47.195 + self.setdefault(n, 0)
47.196 + self[n] += 1
47.197 +
47.198 + def most(self):
47.199 + """Returns the keys with maximum count."""
47.200 + m = max(self.itervalues())
47.201 + return [k for k, v in self.iteritems() if v == m]
47.202 +
47.203 + def least(self):
47.204 + """Returns the keys with mininum count."""
47.205 + m = min(self.itervalues())
47.206 + return [k for k, v in self.iteritems() if v == m]
47.207 +
47.208 + def percent(self, key):
47.209 + """Returns what percentage a certain key is of all entries.
47.210 +
47.211 + >>> c = counter()
47.212 + >>> c.add('x')
47.213 + >>> c.add('x')
47.214 + >>> c.add('x')
47.215 + >>> c.add('y')
47.216 + >>> c.percent('x')
47.217 + 0.75
47.218 + >>> c.percent('y')
47.219 + 0.25
47.220 + """
47.221 + return float(self[key])/sum(self.values())
47.222 +
47.223 + def sorted_keys(self):
47.224 + """Returns keys sorted by value.
47.225 +
47.226 + >>> c = counter()
47.227 + >>> c.add('x')
47.228 + >>> c.add('x')
47.229 + >>> c.add('y')
47.230 + >>> c.sorted_keys()
47.231 + ['x', 'y']
47.232 + """
47.233 + return sorted(self.keys(), key=lambda k: self[k], reverse=True)
47.234 +
47.235 + def sorted_values(self):
47.236 + """Returns values sorted by value.
47.237 +
47.238 + >>> c = counter()
47.239 + >>> c.add('x')
47.240 + >>> c.add('x')
47.241 + >>> c.add('y')
47.242 + >>> c.sorted_values()
47.243 + [2, 1]
47.244 + """
47.245 + return [self[k] for k in self.sorted_keys()]
47.246 +
47.247 + def sorted_items(self):
47.248 + """Returns items sorted by value.
47.249 +
47.250 + >>> c = counter()
47.251 + >>> c.add('x')
47.252 + >>> c.add('x')
47.253 + >>> c.add('y')
47.254 + >>> c.sorted_items()
47.255 + [('x', 2), ('y', 1)]
47.256 + """
47.257 + return [(k, self[k]) for k in self.sorted_keys()]
47.258 +
47.259 + def __repr__(self):
47.260 + return '<Counter ' + dict.__repr__(self) + '>'
47.261 +
47.262 +counter = Counter
47.263 +
47.264 +iters = [list, tuple]
47.265 +import __builtin__
47.266 +if hasattr(__builtin__, 'set'):
47.267 + iters.append(set)
47.268 +if hasattr(__builtin__, 'frozenset'):
47.269 + iters.append(set)
47.270 +if sys.version_info < (2,6): # sets module deprecated in 2.6
47.271 + try:
47.272 + from sets import Set
47.273 + iters.append(Set)
47.274 + except ImportError:
47.275 + pass
47.276 +
47.277 +class _hack(tuple): pass
47.278 +iters = _hack(iters)
47.279 +iters.__doc__ = """
47.280 +A list of iterable items (like lists, but not strings). Includes whichever
47.281 +of lists, tuples, sets, and Sets are available in this version of Python.
47.282 +"""
47.283 +
47.284 +def _strips(direction, text, remove):
47.285 + if isinstance(remove, iters):
47.286 + for subr in remove:
47.287 + text = _strips(direction, text, subr)
47.288 + return text
47.289 +
47.290 + if direction == 'l':
47.291 + if text.startswith(remove):
47.292 + return text[len(remove):]
47.293 + elif direction == 'r':
47.294 + if text.endswith(remove):
47.295 + return text[:-len(remove)]
47.296 + else:
47.297 + raise ValueError, "Direction needs to be r or l."
47.298 + return text
47.299 +
47.300 +def rstrips(text, remove):
47.301 + """
47.302 + removes the string `remove` from the right of `text`
47.303 +
47.304 + >>> rstrips("foobar", "bar")
47.305 + 'foo'
47.306 +
47.307 + """
47.308 + return _strips('r', text, remove)
47.309 +
47.310 +def lstrips(text, remove):
47.311 + """
47.312 + removes the string `remove` from the left of `text`
47.313 +
47.314 + >>> lstrips("foobar", "foo")
47.315 + 'bar'
47.316 + >>> lstrips('http://foo.org/', ['http://', 'https://'])
47.317 + 'foo.org/'
47.318 + >>> lstrips('FOOBARBAZ', ['FOO', 'BAR'])
47.319 + 'BAZ'
47.320 + >>> lstrips('FOOBARBAZ', ['BAR', 'FOO'])
47.321 + 'BARBAZ'
47.322 +
47.323 + """
47.324 + return _strips('l', text, remove)
47.325 +
47.326 +def strips(text, remove):
47.327 + """
47.328 + removes the string `remove` from the both sides of `text`
47.329 +
47.330 + >>> strips("foobarfoo", "foo")
47.331 + 'bar'
47.332 +
47.333 + """
47.334 + return rstrips(lstrips(text, remove), remove)
47.335 +
47.336 +def safeunicode(obj, encoding='utf-8'):
47.337 + r"""
47.338 + Converts any given object to unicode string.
47.339 +
47.340 + >>> safeunicode('hello')
47.341 + u'hello'
47.342 + >>> safeunicode(2)
47.343 + u'2'
47.344 + >>> safeunicode('\xe1\x88\xb4')
47.345 + u'\u1234'
47.346 + """
47.347 + t = type(obj)
47.348 + if t is unicode:
47.349 + return obj
47.350 + elif t is str:
47.351 + return obj.decode(encoding)
47.352 + elif t in [int, float, bool]:
47.353 + return unicode(obj)
47.354 + elif hasattr(obj, '__unicode__') or isinstance(obj, unicode):
47.355 + return unicode(obj)
47.356 + else:
47.357 + return str(obj).decode(encoding)
47.358 +
47.359 +def safestr(obj, encoding='utf-8'):
47.360 + r"""
47.361 + Converts any given object to utf-8 encoded string.
47.362 +
47.363 + >>> safestr('hello')
47.364 + 'hello'
47.365 + >>> safestr(u'\u1234')
47.366 + '\xe1\x88\xb4'
47.367 + >>> safestr(2)
47.368 + '2'
47.369 + """
47.370 + if isinstance(obj, unicode):
47.371 + return obj.encode(encoding)
47.372 + elif isinstance(obj, str):
47.373 + return obj
47.374 + elif hasattr(obj, 'next'): # iterator
47.375 + return itertools.imap(safestr, obj)
47.376 + else:
47.377 + return str(obj)
47.378 +
47.379 +# for backward-compatibility
47.380 +utf8 = safestr
47.381 +
47.382 +class TimeoutError(Exception): pass
47.383 +def timelimit(timeout):
47.384 + """
47.385 + A decorator to limit a function to `timeout` seconds, raising `TimeoutError`
47.386 + if it takes longer.
47.387 +
47.388 + >>> import time
47.389 + >>> def meaningoflife():
47.390 + ... time.sleep(.2)
47.391 + ... return 42
47.392 + >>>
47.393 + >>> timelimit(.1)(meaningoflife)()
47.394 + Traceback (most recent call last):
47.395 + ...
47.396 + TimeoutError: took too long
47.397 + >>> timelimit(1)(meaningoflife)()
47.398 + 42
47.399 +
47.400 + _Caveat:_ The function isn't stopped after `timeout` seconds but continues
47.401 + executing in a separate thread. (There seems to be no way to kill a thread.)
47.402 +
47.403 + inspired by <http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/473878>
47.404 + """
47.405 + def _1(function):
47.406 + def _2(*args, **kw):
47.407 + class Dispatch(threading.Thread):
47.408 + def __init__(self):
47.409 + threading.Thread.__init__(self)
47.410 + self.result = None
47.411 + self.error = None
47.412 +
47.413 + self.setDaemon(True)
47.414 + self.start()
47.415 +
47.416 + def run(self):
47.417 + try:
47.418 + self.result = function(*args, **kw)
47.419 + except:
47.420 + self.error = sys.exc_info()
47.421 +
47.422 + c = Dispatch()
47.423 + c.join(timeout)
47.424 + if c.isAlive():
47.425 + raise TimeoutError, 'took too long'
47.426 + if c.error:
47.427 + raise c.error[0], c.error[1]
47.428 + return c.result
47.429 + return _2
47.430 + return _1
47.431 +
47.432 +class Memoize:
47.433 + """
47.434 + 'Memoizes' a function, caching its return values for each input.
47.435 + If `expires` is specified, values are recalculated after `expires` seconds.
47.436 + If `background` is specified, values are recalculated in a separate thread.
47.437 +
47.438 + >>> calls = 0
47.439 + >>> def howmanytimeshaveibeencalled():
47.440 + ... global calls
47.441 + ... calls += 1
47.442 + ... return calls
47.443 + >>> fastcalls = memoize(howmanytimeshaveibeencalled)
47.444 + >>> howmanytimeshaveibeencalled()
47.445 + 1
47.446 + >>> howmanytimeshaveibeencalled()
47.447 + 2
47.448 + >>> fastcalls()
47.449 + 3
47.450 + >>> fastcalls()
47.451 + 3
47.452 + >>> import time
47.453 + >>> fastcalls = memoize(howmanytimeshaveibeencalled, .1, background=False)
47.454 + >>> fastcalls()
47.455 + 4
47.456 + >>> fastcalls()
47.457 + 4
47.458 + >>> time.sleep(.2)
47.459 + >>> fastcalls()
47.460 + 5
47.461 + >>> def slowfunc():
47.462 + ... time.sleep(.1)
47.463 + ... return howmanytimeshaveibeencalled()
47.464 + >>> fastcalls = memoize(slowfunc, .2, background=True)
47.465 + >>> fastcalls()
47.466 + 6
47.467 + >>> timelimit(.05)(fastcalls)()
47.468 + 6
47.469 + >>> time.sleep(.2)
47.470 + >>> timelimit(.05)(fastcalls)()
47.471 + 6
47.472 + >>> timelimit(.05)(fastcalls)()
47.473 + 6
47.474 + >>> time.sleep(.2)
47.475 + >>> timelimit(.05)(fastcalls)()
47.476 + 7
47.477 + >>> fastcalls = memoize(slowfunc, None, background=True)
47.478 + >>> threading.Thread(target=fastcalls).start()
47.479 + >>> time.sleep(.01)
47.480 + >>> fastcalls()
47.481 + 9
47.482 + """
47.483 + def __init__(self, func, expires=None, background=True):
47.484 + self.func = func
47.485 + self.cache = {}
47.486 + self.expires = expires
47.487 + self.background = background
47.488 + self.running = {}
47.489 +
47.490 + def __call__(self, *args, **keywords):
47.491 + key = (args, tuple(keywords.items()))
47.492 + if not self.running.get(key):
47.493 + self.running[key] = threading.Lock()
47.494 + def update(block=False):
47.495 + if self.running[key].acquire(block):
47.496 + try:
47.497 + self.cache[key] = (self.func(*args, **keywords), time.time())
47.498 + finally:
47.499 + self.running[key].release()
47.500 +
47.501 + if key not in self.cache:
47.502 + update(block=True)
47.503 + elif self.expires and (time.time() - self.cache[key][1]) > self.expires:
47.504 + if self.background:
47.505 + threading.Thread(target=update).start()
47.506 + else:
47.507 + update()
47.508 + return self.cache[key][0]
47.509 +
47.510 +memoize = Memoize
47.511 +
47.512 +re_compile = memoize(re.compile) #@@ threadsafe?
47.513 +re_compile.__doc__ = """
47.514 +A memoized version of re.compile.
47.515 +"""
47.516 +
47.517 +class _re_subm_proxy:
47.518 + def __init__(self):
47.519 + self.match = None
47.520 + def __call__(self, match):
47.521 + self.match = match
47.522 + return ''
47.523 +
47.524 +def re_subm(pat, repl, string):
47.525 + """
47.526 + Like re.sub, but returns the replacement _and_ the match object.
47.527 +
47.528 + >>> t, m = re_subm('g(oo+)fball', r'f\\1lish', 'goooooofball')
47.529 + >>> t
47.530 + 'foooooolish'
47.531 + >>> m.groups()
47.532 + ('oooooo',)
47.533 + """
47.534 + compiled_pat = re_compile(pat)
47.535 + proxy = _re_subm_proxy()
47.536 + compiled_pat.sub(proxy.__call__, string)
47.537 + return compiled_pat.sub(repl, string), proxy.match
47.538 +
47.539 +def group(seq, size):
47.540 + """
47.541 + Returns an iterator over a series of lists of length size from iterable.
47.542 +
47.543 + >>> list(group([1,2,3,4], 2))
47.544 + [[1, 2], [3, 4]]
47.545 + >>> list(group([1,2,3,4,5], 2))
47.546 + [[1, 2], [3, 4], [5]]
47.547 + """
47.548 + def take(seq, n):
47.549 + for i in xrange(n):
47.550 + yield seq.next()
47.551 +
47.552 + if not hasattr(seq, 'next'):
47.553 + seq = iter(seq)
47.554 + while True:
47.555 + x = list(take(seq, size))
47.556 + if x:
47.557 + yield x
47.558 + else:
47.559 + break
47.560 +
47.561 +def uniq(seq, key=None):
47.562 + """
47.563 + Removes duplicate elements from a list while preserving the order of the rest.
47.564 +
47.565 + >>> uniq([9,0,2,1,0])
47.566 + [9, 0, 2, 1]
47.567 +
47.568 + The value of the optional `key` parameter should be a function that
47.569 + takes a single argument and returns a key to test the uniqueness.
47.570 +
47.571 + >>> uniq(["Foo", "foo", "bar"], key=lambda s: s.lower())
47.572 + ['Foo', 'bar']
47.573 + """
47.574 + key = key or (lambda x: x)
47.575 + seen = set()
47.576 + result = []
47.577 + for v in seq:
47.578 + k = key(v)
47.579 + if k in seen:
47.580 + continue
47.581 + seen.add(k)
47.582 + result.append(v)
47.583 + return result
47.584 +
47.585 +def iterview(x):
47.586 + """
47.587 + Takes an iterable `x` and returns an iterator over it
47.588 + which prints its progress to stderr as it iterates through.
47.589 + """
47.590 + WIDTH = 70
47.591 +
47.592 + def plainformat(n, lenx):
47.593 + return '%5.1f%% (%*d/%d)' % ((float(n)/lenx)*100, len(str(lenx)), n, lenx)
47.594 +
47.595 + def bars(size, n, lenx):
47.596 + val = int((float(n)*size)/lenx + 0.5)
47.597 + if size - val:
47.598 + spacing = ">" + (" "*(size-val))[1:]
47.599 + else:
47.600 + spacing = ""
47.601 + return "[%s%s]" % ("="*val, spacing)
47.602 +
47.603 + def eta(elapsed, n, lenx):
47.604 + if n == 0:
47.605 + return '--:--:--'
47.606 + if n == lenx:
47.607 + secs = int(elapsed)
47.608 + else:
47.609 + secs = int((elapsed/n) * (lenx-n))
47.610 + mins, secs = divmod(secs, 60)
47.611 + hrs, mins = divmod(mins, 60)
47.612 +
47.613 + return '%02d:%02d:%02d' % (hrs, mins, secs)
47.614 +
47.615 + def format(starttime, n, lenx):
47.616 + out = plainformat(n, lenx) + ' '
47.617 + if n == lenx:
47.618 + end = ' '
47.619 + else:
47.620 + end = ' ETA '
47.621 + end += eta(time.time() - starttime, n, lenx)
47.622 + out += bars(WIDTH - len(out) - len(end), n, lenx)
47.623 + out += end
47.624 + return out
47.625 +
47.626 + starttime = time.time()
47.627 + lenx = len(x)
47.628 + for n, y in enumerate(x):
47.629 + sys.stderr.write('\r' + format(starttime, n, lenx))
47.630 + yield y
47.631 + sys.stderr.write('\r' + format(starttime, n+1, lenx) + '\n')
47.632 +
47.633 +class IterBetter:
47.634 + """
47.635 + Returns an object that can be used as an iterator
47.636 + but can also be used via __getitem__ (although it
47.637 + cannot go backwards -- that is, you cannot request
47.638 + `iterbetter[0]` after requesting `iterbetter[1]`).
47.639 +
47.640 + >>> import itertools
47.641 + >>> c = iterbetter(itertools.count())
47.642 + >>> c[1]
47.643 + 1
47.644 + >>> c[5]
47.645 + 5
47.646 + >>> c[3]
47.647 + Traceback (most recent call last):
47.648 + ...
47.649 + IndexError: already passed 3
47.650 +
47.651 + For boolean test, IterBetter peeps at first value in the itertor without effecting the iteration.
47.652 +
47.653 + >>> c = iterbetter(iter(range(5)))
47.654 + >>> bool(c)
47.655 + True
47.656 + >>> list(c)
47.657 + [0, 1, 2, 3, 4]
47.658 + >>> c = iterbetter(iter([]))
47.659 + >>> bool(c)
47.660 + False
47.661 + >>> list(c)
47.662 + []
47.663 + """
47.664 + def __init__(self, iterator):
47.665 + self.i, self.c = iterator, 0
47.666 +
47.667 + def __iter__(self):
47.668 + if hasattr(self, "_head"):
47.669 + yield self._head
47.670 +
47.671 + while 1:
47.672 + yield self.i.next()
47.673 + self.c += 1
47.674 +
47.675 + def __getitem__(self, i):
47.676 + #todo: slices
47.677 + if i < self.c:
47.678 + raise IndexError, "already passed "+str(i)
47.679 + try:
47.680 + while i > self.c:
47.681 + self.i.next()
47.682 + self.c += 1
47.683 + # now self.c == i
47.684 + self.c += 1
47.685 + return self.i.next()
47.686 + except StopIteration:
47.687 + raise IndexError, str(i)
47.688 +
47.689 + def __nonzero__(self):
47.690 + if hasattr(self, "__len__"):
47.691 + return len(self) != 0
47.692 + elif hasattr(self, "_head"):
47.693 + return True
47.694 + else:
47.695 + try:
47.696 + self._head = self.i.next()
47.697 + except StopIteration:
47.698 + return False
47.699 + else:
47.700 + return True
47.701 +
47.702 +iterbetter = IterBetter
47.703 +
47.704 +def safeiter(it, cleanup=None, ignore_errors=True):
47.705 + """Makes an iterator safe by ignoring the exceptions occured during the iteration.
47.706 + """
47.707 + def next():
47.708 + while True:
47.709 + try:
47.710 + return it.next()
47.711 + except StopIteration:
47.712 + raise
47.713 + except:
47.714 + traceback.print_exc()
47.715 +
47.716 + it = iter(it)
47.717 + while True:
47.718 + yield next()
47.719 +
47.720 +def safewrite(filename, content):
47.721 + """Writes the content to a temp file and then moves the temp file to
47.722 + given filename to avoid overwriting the existing file in case of errors.
47.723 + """
47.724 + f = file(filename + '.tmp', 'w')
47.725 + f.write(content)
47.726 + f.close()
47.727 + os.rename(f.name, filename)
47.728 +
47.729 +def dictreverse(mapping):
47.730 + """
47.731 + Returns a new dictionary with keys and values swapped.
47.732 +
47.733 + >>> dictreverse({1: 2, 3: 4})
47.734 + {2: 1, 4: 3}
47.735 + """
47.736 + return dict([(value, key) for (key, value) in mapping.iteritems()])
47.737 +
47.738 +def dictfind(dictionary, element):
47.739 + """
47.740 + Returns a key whose value in `dictionary` is `element`
47.741 + or, if none exists, None.
47.742 +
47.743 + >>> d = {1:2, 3:4}
47.744 + >>> dictfind(d, 4)
47.745 + 3
47.746 + >>> dictfind(d, 5)
47.747 + """
47.748 + for (key, value) in dictionary.iteritems():
47.749 + if element is value:
47.750 + return key
47.751 +
47.752 +def dictfindall(dictionary, element):
47.753 + """
47.754 + Returns the keys whose values in `dictionary` are `element`
47.755 + or, if none exists, [].
47.756 +
47.757 + >>> d = {1:4, 3:4}
47.758 + >>> dictfindall(d, 4)
47.759 + [1, 3]
47.760 + >>> dictfindall(d, 5)
47.761 + []
47.762 + """
47.763 + res = []
47.764 + for (key, value) in dictionary.iteritems():
47.765 + if element is value:
47.766 + res.append(key)
47.767 + return res
47.768 +
47.769 +def dictincr(dictionary, element):
47.770 + """
47.771 + Increments `element` in `dictionary`,
47.772 + setting it to one if it doesn't exist.
47.773 +
47.774 + >>> d = {1:2, 3:4}
47.775 + >>> dictincr(d, 1)
47.776 + 3
47.777 + >>> d[1]
47.778 + 3
47.779 + >>> dictincr(d, 5)
47.780 + 1
47.781 + >>> d[5]
47.782 + 1
47.783 + """
47.784 + dictionary.setdefault(element, 0)
47.785 + dictionary[element] += 1
47.786 + return dictionary[element]
47.787 +
47.788 +def dictadd(*dicts):
47.789 + """
47.790 + Returns a dictionary consisting of the keys in the argument dictionaries.
47.791 + If they share a key, the value from the last argument is used.
47.792 +
47.793 + >>> dictadd({1: 0, 2: 0}, {2: 1, 3: 1})
47.794 + {1: 0, 2: 1, 3: 1}
47.795 + """
47.796 + result = {}
47.797 + for dct in dicts:
47.798 + result.update(dct)
47.799 + return result
47.800 +
47.801 +def requeue(queue, index=-1):
47.802 + """Returns the element at index after moving it to the beginning of the queue.
47.803 +
47.804 + >>> x = [1, 2, 3, 4]
47.805 + >>> requeue(x)
47.806 + 4
47.807 + >>> x
47.808 + [4, 1, 2, 3]
47.809 + """
47.810 + x = queue.pop(index)
47.811 + queue.insert(0, x)
47.812 + return x
47.813 +
47.814 +def restack(stack, index=0):
47.815 + """Returns the element at index after moving it to the top of stack.
47.816 +
47.817 + >>> x = [1, 2, 3, 4]
47.818 + >>> restack(x)
47.819 + 1
47.820 + >>> x
47.821 + [2, 3, 4, 1]
47.822 + """
47.823 + x = stack.pop(index)
47.824 + stack.append(x)
47.825 + return x
47.826 +
47.827 +def listget(lst, ind, default=None):
47.828 + """
47.829 + Returns `lst[ind]` if it exists, `default` otherwise.
47.830 +
47.831 + >>> listget(['a'], 0)
47.832 + 'a'
47.833 + >>> listget(['a'], 1)
47.834 + >>> listget(['a'], 1, 'b')
47.835 + 'b'
47.836 + """
47.837 + if len(lst)-1 < ind:
47.838 + return default
47.839 + return lst[ind]
47.840 +
47.841 +def intget(integer, default=None):
47.842 + """
47.843 + Returns `integer` as an int or `default` if it can't.
47.844 +
47.845 + >>> intget('3')
47.846 + 3
47.847 + >>> intget('3a')
47.848 + >>> intget('3a', 0)
47.849 + 0
47.850 + """
47.851 + try:
47.852 + return int(integer)
47.853 + except (TypeError, ValueError):
47.854 + return default
47.855 +
47.856 +def datestr(then, now=None):
47.857 + """
47.858 + Converts a (UTC) datetime object to a nice string representation.
47.859 +
47.860 + >>> from datetime import datetime, timedelta
47.861 + >>> d = datetime(1970, 5, 1)
47.862 + >>> datestr(d, now=d)
47.863 + '0 microseconds ago'
47.864 + >>> for t, v in {
47.865 + ... timedelta(microseconds=1): '1 microsecond ago',
47.866 + ... timedelta(microseconds=2): '2 microseconds ago',
47.867 + ... -timedelta(microseconds=1): '1 microsecond from now',
47.868 + ... -timedelta(microseconds=2): '2 microseconds from now',
47.869 + ... timedelta(microseconds=2000): '2 milliseconds ago',
47.870 + ... timedelta(seconds=2): '2 seconds ago',
47.871 + ... timedelta(seconds=2*60): '2 minutes ago',
47.872 + ... timedelta(seconds=2*60*60): '2 hours ago',
47.873 + ... timedelta(days=2): '2 days ago',
47.874 + ... }.iteritems():
47.875 + ... assert datestr(d, now=d+t) == v
47.876 + >>> datestr(datetime(1970, 1, 1), now=d)
47.877 + 'January 1'
47.878 + >>> datestr(datetime(1969, 1, 1), now=d)
47.879 + 'January 1, 1969'
47.880 + >>> datestr(datetime(1970, 6, 1), now=d)
47.881 + 'June 1, 1970'
47.882 + >>> datestr(None)
47.883 + ''
47.884 + """
47.885 + def agohence(n, what, divisor=None):
47.886 + if divisor: n = n // divisor
47.887 +
47.888 + out = str(abs(n)) + ' ' + what # '2 day'
47.889 + if abs(n) != 1: out += 's' # '2 days'
47.890 + out += ' ' # '2 days '
47.891 + if n < 0:
47.892 + out += 'from now'
47.893 + else:
47.894 + out += 'ago'
47.895 + return out # '2 days ago'
47.896 +
47.897 + oneday = 24 * 60 * 60
47.898 +
47.899 + if not then: return ""
47.900 + if not now: now = datetime.datetime.utcnow()
47.901 + if type(now).__name__ == "DateTime":
47.902 + now = datetime.datetime.fromtimestamp(now)
47.903 + if type(then).__name__ == "DateTime":
47.904 + then = datetime.datetime.fromtimestamp(then)
47.905 + elif type(then).__name__ == "date":
47.906 + then = datetime.datetime(then.year, then.month, then.day)
47.907 +
47.908 + delta = now - then
47.909 + deltaseconds = int(delta.days * oneday + delta.seconds + delta.microseconds * 1e-06)
47.910 + deltadays = abs(deltaseconds) // oneday
47.911 + if deltaseconds < 0: deltadays *= -1 # fix for oddity of floor
47.912 +
47.913 + if deltadays:
47.914 + if abs(deltadays) < 4:
47.915 + return agohence(deltadays, 'day')
47.916 +
47.917 + try:
47.918 + out = then.strftime('%B %e') # e.g. 'June 3'
47.919 + except ValueError:
47.920 + # %e doesn't work on Windows.
47.921 + out = then.strftime('%B %d') # e.g. 'June 03'
47.922 +
47.923 + if then.year != now.year or deltadays < 0:
47.924 + out += ', %s' % then.year
47.925 + return out
47.926 +
47.927 + if int(deltaseconds):
47.928 + if abs(deltaseconds) > (60 * 60):
47.929 + return agohence(deltaseconds, 'hour', 60 * 60)
47.930 + elif abs(deltaseconds) > 60:
47.931 + return agohence(deltaseconds, 'minute', 60)
47.932 + else:
47.933 + return agohence(deltaseconds, 'second')
47.934 +
47.935 + deltamicroseconds = delta.microseconds
47.936 + if delta.days: deltamicroseconds = int(delta.microseconds - 1e6) # datetime oddity
47.937 + if abs(deltamicroseconds) > 1000:
47.938 + return agohence(deltamicroseconds, 'millisecond', 1000)
47.939 +
47.940 + return agohence(deltamicroseconds, 'microsecond')
47.941 +
47.942 +def numify(string):
47.943 + """
47.944 + Removes all non-digit characters from `string`.
47.945 +
47.946 + >>> numify('800-555-1212')
47.947 + '8005551212'
47.948 + >>> numify('800.555.1212')
47.949 + '8005551212'
47.950 +
47.951 + """
47.952 + return ''.join([c for c in str(string) if c.isdigit()])
47.953 +
47.954 +def denumify(string, pattern):
47.955 + """
47.956 + Formats `string` according to `pattern`, where the letter X gets replaced
47.957 + by characters from `string`.
47.958 +
47.959 + >>> denumify("8005551212", "(XXX) XXX-XXXX")
47.960 + '(800) 555-1212'
47.961 +
47.962 + """
47.963 + out = []
47.964 + for c in pattern:
47.965 + if c == "X":
47.966 + out.append(string[0])
47.967 + string = string[1:]
47.968 + else:
47.969 + out.append(c)
47.970 + return ''.join(out)
47.971 +
47.972 +def commify(n):
47.973 + """
47.974 + Add commas to an integer `n`.
47.975 +
47.976 + >>> commify(1)
47.977 + '1'
47.978 + >>> commify(123)
47.979 + '123'
47.980 + >>> commify(1234)
47.981 + '1,234'
47.982 + >>> commify(1234567890)
47.983 + '1,234,567,890'
47.984 + >>> commify(123.0)
47.985 + '123.0'
47.986 + >>> commify(1234.5)
47.987 + '1,234.5'
47.988 + >>> commify(1234.56789)
47.989 + '1,234.56789'
47.990 + >>> commify('%.2f' % 1234.5)
47.991 + '1,234.50'
47.992 + >>> commify(None)
47.993 + >>>
47.994 +
47.995 + """
47.996 + if n is None: return None
47.997 + n = str(n)
47.998 + if '.' in n:
47.999 + dollars, cents = n.split('.')
47.1000 + else:
47.1001 + dollars, cents = n, None
47.1002 +
47.1003 + r = []
47.1004 + for i, c in enumerate(str(dollars)[::-1]):
47.1005 + if i and (not (i % 3)):
47.1006 + r.insert(0, ',')
47.1007 + r.insert(0, c)
47.1008 + out = ''.join(r)
47.1009 + if cents:
47.1010 + out += '.' + cents
47.1011 + return out
47.1012 +
47.1013 +def dateify(datestring):
47.1014 + """
47.1015 + Formats a numified `datestring` properly.
47.1016 + """
47.1017 + return denumify(datestring, "XXXX-XX-XX XX:XX:XX")
47.1018 +
47.1019 +
47.1020 +def nthstr(n):
47.1021 + """
47.1022 + Formats an ordinal.
47.1023 + Doesn't handle negative numbers.
47.1024 +
47.1025 + >>> nthstr(1)
47.1026 + '1st'
47.1027 + >>> nthstr(0)
47.1028 + '0th'
47.1029 + >>> [nthstr(x) for x in [2, 3, 4, 5, 10, 11, 12, 13, 14, 15]]
47.1030 + ['2nd', '3rd', '4th', '5th', '10th', '11th', '12th', '13th', '14th', '15th']
47.1031 + >>> [nthstr(x) for x in [91, 92, 93, 94, 99, 100, 101, 102]]
47.1032 + ['91st', '92nd', '93rd', '94th', '99th', '100th', '101st', '102nd']
47.1033 + >>> [nthstr(x) for x in [111, 112, 113, 114, 115]]
47.1034 + ['111th', '112th', '113th', '114th', '115th']
47.1035 +
47.1036 + """
47.1037 +
47.1038 + assert n >= 0
47.1039 + if n % 100 in [11, 12, 13]: return '%sth' % n
47.1040 + return {1: '%sst', 2: '%snd', 3: '%srd'}.get(n % 10, '%sth') % n
47.1041 +
47.1042 +def cond(predicate, consequence, alternative=None):
47.1043 + """
47.1044 + Function replacement for if-else to use in expressions.
47.1045 +
47.1046 + >>> x = 2
47.1047 + >>> cond(x % 2 == 0, "even", "odd")
47.1048 + 'even'
47.1049 + >>> cond(x % 2 == 0, "even", "odd") + '_row'
47.1050 + 'even_row'
47.1051 + """
47.1052 + if predicate:
47.1053 + return consequence
47.1054 + else:
47.1055 + return alternative
47.1056 +
47.1057 +class CaptureStdout:
47.1058 + """
47.1059 + Captures everything `func` prints to stdout and returns it instead.
47.1060 +
47.1061 + >>> def idiot():
47.1062 + ... print "foo"
47.1063 + >>> capturestdout(idiot)()
47.1064 + 'foo\\n'
47.1065 +
47.1066 + **WARNING:** Not threadsafe!
47.1067 + """
47.1068 + def __init__(self, func):
47.1069 + self.func = func
47.1070 + def __call__(self, *args, **keywords):
47.1071 + from cStringIO import StringIO
47.1072 + # Not threadsafe!
47.1073 + out = StringIO()
47.1074 + oldstdout = sys.stdout
47.1075 + sys.stdout = out
47.1076 + try:
47.1077 + self.func(*args, **keywords)
47.1078 + finally:
47.1079 + sys.stdout = oldstdout
47.1080 + return out.getvalue()
47.1081 +
47.1082 +capturestdout = CaptureStdout
47.1083 +
47.1084 +class Profile:
47.1085 + """
47.1086 + Profiles `func` and returns a tuple containing its output
47.1087 + and a string with human-readable profiling information.
47.1088 +
47.1089 + >>> import time
47.1090 + >>> out, inf = profile(time.sleep)(.001)
47.1091 + >>> out
47.1092 + >>> inf[:10].strip()
47.1093 + 'took 0.0'
47.1094 + """
47.1095 + def __init__(self, func):
47.1096 + self.func = func
47.1097 + def __call__(self, *args): ##, **kw): kw unused
47.1098 + import hotshot, hotshot.stats, os, tempfile ##, time already imported
47.1099 + f, filename = tempfile.mkstemp()
47.1100 + os.close(f)
47.1101 +
47.1102 + prof = hotshot.Profile(filename)
47.1103 +
47.1104 + stime = time.time()
47.1105 + result = prof.runcall(self.func, *args)
47.1106 + stime = time.time() - stime
47.1107 + prof.close()
47.1108 +
47.1109 + import cStringIO
47.1110 + out = cStringIO.StringIO()
47.1111 + stats = hotshot.stats.load(filename)
47.1112 + stats.stream = out
47.1113 + stats.strip_dirs()
47.1114 + stats.sort_stats('time', 'calls')
47.1115 + stats.print_stats(40)
47.1116 + stats.print_callers()
47.1117 +
47.1118 + x = '\n\ntook '+ str(stime) + ' seconds\n'
47.1119 + x += out.getvalue()
47.1120 +
47.1121 + # remove the tempfile
47.1122 + try:
47.1123 + os.remove(filename)
47.1124 + except IOError:
47.1125 + pass
47.1126 +
47.1127 + return result, x
47.1128 +
47.1129 +profile = Profile
47.1130 +
47.1131 +
47.1132 +import traceback
47.1133 +# hack for compatibility with Python 2.3:
47.1134 +if not hasattr(traceback, 'format_exc'):
47.1135 + from cStringIO import StringIO
47.1136 + def format_exc(limit=None):
47.1137 + strbuf = StringIO()
47.1138 + traceback.print_exc(limit, strbuf)
47.1139 + return strbuf.getvalue()
47.1140 + traceback.format_exc = format_exc
47.1141 +
47.1142 +def tryall(context, prefix=None):
47.1143 + """
47.1144 + Tries a series of functions and prints their results.
47.1145 + `context` is a dictionary mapping names to values;
47.1146 + the value will only be tried if it's callable.
47.1147 +
47.1148 + >>> tryall(dict(j=lambda: True))
47.1149 + j: True
47.1150 + ----------------------------------------
47.1151 + results:
47.1152 + True: 1
47.1153 +
47.1154 + For example, you might have a file `test/stuff.py`
47.1155 + with a series of functions testing various things in it.
47.1156 + At the bottom, have a line:
47.1157 +
47.1158 + if __name__ == "__main__": tryall(globals())
47.1159 +
47.1160 + Then you can run `python test/stuff.py` and get the results of
47.1161 + all the tests.
47.1162 + """
47.1163 + context = context.copy() # vars() would update
47.1164 + results = {}
47.1165 + for (key, value) in context.iteritems():
47.1166 + if not hasattr(value, '__call__'):
47.1167 + continue
47.1168 + if prefix and not key.startswith(prefix):
47.1169 + continue
47.1170 + print key + ':',
47.1171 + try:
47.1172 + r = value()
47.1173 + dictincr(results, r)
47.1174 + print r
47.1175 + except:
47.1176 + print 'ERROR'
47.1177 + dictincr(results, 'ERROR')
47.1178 + print ' ' + '\n '.join(traceback.format_exc().split('\n'))
47.1179 +
47.1180 + print '-'*40
47.1181 + print 'results:'
47.1182 + for (key, value) in results.iteritems():
47.1183 + print ' '*2, str(key)+':', value
47.1184 +
47.1185 +class ThreadedDict(threadlocal):
47.1186 + """
47.1187 + Thread local storage.
47.1188 +
47.1189 + >>> d = ThreadedDict()
47.1190 + >>> d.x = 1
47.1191 + >>> d.x
47.1192 + 1
47.1193 + >>> import threading
47.1194 + >>> def f(): d.x = 2
47.1195 + ...
47.1196 + >>> t = threading.Thread(target=f)
47.1197 + >>> t.start()
47.1198 + >>> t.join()
47.1199 + >>> d.x
47.1200 + 1
47.1201 + """
47.1202 + _instances = set()
47.1203 +
47.1204 + def __init__(self):
47.1205 + ThreadedDict._instances.add(self)
47.1206 +
47.1207 + def __del__(self):
47.1208 + ThreadedDict._instances.remove(self)
47.1209 +
47.1210 + def __hash__(self):
47.1211 + return id(self)
47.1212 +
47.1213 + def clear_all():
47.1214 + """Clears all ThreadedDict instances.
47.1215 + """
47.1216 + for t in list(ThreadedDict._instances):
47.1217 + t.clear()
47.1218 + clear_all = staticmethod(clear_all)
47.1219 +
47.1220 + # Define all these methods to more or less fully emulate dict -- attribute access
47.1221 + # is built into threading.local.
47.1222 +
47.1223 + def __getitem__(self, key):
47.1224 + return self.__dict__[key]
47.1225 +
47.1226 + def __setitem__(self, key, value):
47.1227 + self.__dict__[key] = value
47.1228 +
47.1229 + def __delitem__(self, key):
47.1230 + del self.__dict__[key]
47.1231 +
47.1232 + def __contains__(self, key):
47.1233 + return key in self.__dict__
47.1234 +
47.1235 + has_key = __contains__
47.1236 +
47.1237 + def clear(self):
47.1238 + self.__dict__.clear()
47.1239 +
47.1240 + def copy(self):
47.1241 + return self.__dict__.copy()
47.1242 +
47.1243 + def get(self, key, default=None):
47.1244 + return self.__dict__.get(key, default)
47.1245 +
47.1246 + def items(self):
47.1247 + return self.__dict__.items()
47.1248 +
47.1249 + def iteritems(self):
47.1250 + return self.__dict__.iteritems()
47.1251 +
47.1252 + def keys(self):
47.1253 + return self.__dict__.keys()
47.1254 +
47.1255 + def iterkeys(self):
47.1256 + return self.__dict__.iterkeys()
47.1257 +
47.1258 + iter = iterkeys
47.1259 +
47.1260 + def values(self):
47.1261 + return self.__dict__.values()
47.1262 +
47.1263 + def itervalues(self):
47.1264 + return self.__dict__.itervalues()
47.1265 +
47.1266 + def pop(self, key, *args):
47.1267 + return self.__dict__.pop(key, *args)
47.1268 +
47.1269 + def popitem(self):
47.1270 + return self.__dict__.popitem()
47.1271 +
47.1272 + def setdefault(self, key, default=None):
47.1273 + return self.__dict__.setdefault(key, default)
47.1274 +
47.1275 + def update(self, *args, **kwargs):
47.1276 + self.__dict__.update(*args, **kwargs)
47.1277 +
47.1278 + def __repr__(self):
47.1279 + return '<ThreadedDict %r>' % self.__dict__
47.1280 +
47.1281 + __str__ = __repr__
47.1282 +
47.1283 +threadeddict = ThreadedDict
47.1284 +
47.1285 +def autoassign(self, locals):
47.1286 + """
47.1287 + Automatically assigns local variables to `self`.
47.1288 +
47.1289 + >>> self = storage()
47.1290 + >>> autoassign(self, dict(a=1, b=2))
47.1291 + >>> self
47.1292 + <Storage {'a': 1, 'b': 2}>
47.1293 +
47.1294 + Generally used in `__init__` methods, as in:
47.1295 +
47.1296 + def __init__(self, foo, bar, baz=1): autoassign(self, locals())
47.1297 + """
47.1298 + for (key, value) in locals.iteritems():
47.1299 + if key == 'self':
47.1300 + continue
47.1301 + setattr(self, key, value)
47.1302 +
47.1303 +def to36(q):
47.1304 + """
47.1305 + Converts an integer to base 36 (a useful scheme for human-sayable IDs).
47.1306 +
47.1307 + >>> to36(35)
47.1308 + 'z'
47.1309 + >>> to36(119292)
47.1310 + '2k1o'
47.1311 + >>> int(to36(939387374), 36)
47.1312 + 939387374
47.1313 + >>> to36(0)
47.1314 + '0'
47.1315 + >>> to36(-393)
47.1316 + Traceback (most recent call last):
47.1317 + ...
47.1318 + ValueError: must supply a positive integer
47.1319 +
47.1320 + """
47.1321 + if q < 0: raise ValueError, "must supply a positive integer"
47.1322 + letters = "0123456789abcdefghijklmnopqrstuvwxyz"
47.1323 + converted = []
47.1324 + while q != 0:
47.1325 + q, r = divmod(q, 36)
47.1326 + converted.insert(0, letters[r])
47.1327 + return "".join(converted) or '0'
47.1328 +
47.1329 +
47.1330 +r_url = re_compile('(?<!\()(http://(\S+))')
47.1331 +def safemarkdown(text):
47.1332 + """
47.1333 + Converts text to HTML following the rules of Markdown, but blocking any
47.1334 + outside HTML input, so that only the things supported by Markdown
47.1335 + can be used. Also converts raw URLs to links.
47.1336 +
47.1337 + (requires [markdown.py](http://webpy.org/markdown.py))
47.1338 + """
47.1339 + from markdown import markdown
47.1340 + if text:
47.1341 + text = text.replace('<', '<')
47.1342 + # TODO: automatically get page title?
47.1343 + text = r_url.sub(r'<\1>', text)
47.1344 + text = markdown(text)
47.1345 + return text
47.1346 +
47.1347 +def sendmail(from_address, to_address, subject, message, headers=None, **kw):
47.1348 + """
47.1349 + Sends the email message `message` with mail and envelope headers
47.1350 + for from `from_address_` to `to_address` with `subject`.
47.1351 + Additional email headers can be specified with the dictionary
47.1352 + `headers.
47.1353 +
47.1354 + Optionally cc, bcc and attachments can be specified as keyword arguments.
47.1355 + Attachments must be an iterable and each attachment can be either a
47.1356 + filename or a file object or a dictionary with filename, content and
47.1357 + optionally content_type keys.
47.1358 +
47.1359 + If `web.config.smtp_server` is set, it will send the message
47.1360 + to that SMTP server. Otherwise it will look for
47.1361 + `/usr/sbin/sendmail`, the typical location for the sendmail-style
47.1362 + binary. To use sendmail from a different path, set `web.config.sendmail_path`.
47.1363 + """
47.1364 + attachments = kw.pop("attachments", [])
47.1365 + mail = _EmailMessage(from_address, to_address, subject, message, headers, **kw)
47.1366 +
47.1367 + for a in attachments:
47.1368 + if isinstance(a, dict):
47.1369 + mail.attach(a['filename'], a['content'], a.get('content_type'))
47.1370 + elif hasattr(a, 'read'): # file
47.1371 + filename = os.path.basename(getattr(a, "name", ""))
47.1372 + content_type = getattr(a, 'content_type', None)
47.1373 + mail.attach(filename, a.read(), content_type)
47.1374 + elif isinstance(a, basestring):
47.1375 + f = open(a, 'rb')
47.1376 + content = f.read()
47.1377 + f.close()
47.1378 + filename = os.path.basename(a)
47.1379 + mail.attach(filename, content, None)
47.1380 + else:
47.1381 + raise ValueError, "Invalid attachment: %s" % repr(a)
47.1382 +
47.1383 + mail.send()
47.1384 +
47.1385 +class _EmailMessage:
47.1386 + def __init__(self, from_address, to_address, subject, message, headers=None, **kw):
47.1387 + def listify(x):
47.1388 + if not isinstance(x, list):
47.1389 + return [safestr(x)]
47.1390 + else:
47.1391 + return [safestr(a) for a in x]
47.1392 +
47.1393 + subject = safestr(subject)
47.1394 + message = safestr(message)
47.1395 +
47.1396 + from_address = safestr(from_address)
47.1397 + to_address = listify(to_address)
47.1398 + cc = listify(kw.get('cc', []))
47.1399 + bcc = listify(kw.get('bcc', []))
47.1400 + recipients = to_address + cc + bcc
47.1401 +
47.1402 + import email.Utils
47.1403 + self.from_address = email.Utils.parseaddr(from_address)[1]
47.1404 + self.recipients = [email.Utils.parseaddr(r)[1] for r in recipients]
47.1405 +
47.1406 + self.headers = dictadd({
47.1407 + 'From': from_address,
47.1408 + 'To': ", ".join(to_address),
47.1409 + 'Subject': subject
47.1410 + }, headers or {})
47.1411 +
47.1412 + if cc:
47.1413 + self.headers['Cc'] = ", ".join(cc)
47.1414 +
47.1415 + self.message = self.new_message()
47.1416 + self.message.add_header("Content-Transfer-Encoding", "7bit")
47.1417 + self.message.add_header("Content-Disposition", "inline")
47.1418 + self.message.add_header("MIME-Version", "1.0")
47.1419 + self.message.set_payload(message, 'utf-8')
47.1420 + self.multipart = False
47.1421 +
47.1422 + def new_message(self):
47.1423 + from email.Message import Message
47.1424 + return Message()
47.1425 +
47.1426 + def attach(self, filename, content, content_type=None):
47.1427 + if not self.multipart:
47.1428 + msg = self.new_message()
47.1429 + msg.add_header("Content-Type", "multipart/mixed")
47.1430 + msg.attach(self.message)
47.1431 + self.message = msg
47.1432 + self.multipart = True
47.1433 +
47.1434 + import mimetypes
47.1435 + try:
47.1436 + from email import encoders
47.1437 + except:
47.1438 + from email import Encoders as encoders
47.1439 +
47.1440 + content_type = content_type or mimetypes.guess_type(filename)[0] or "applcation/octet-stream"
47.1441 +
47.1442 + msg = self.new_message()
47.1443 + msg.set_payload(content)
47.1444 + msg.add_header('Content-Type', content_type)
47.1445 + msg.add_header('Content-Disposition', 'attachment', filename=filename)
47.1446 +
47.1447 + if not content_type.startswith("text/"):
47.1448 + encoders.encode_base64(msg)
47.1449 +
47.1450 + self.message.attach(msg)
47.1451 +
47.1452 + def prepare_message(self):
47.1453 + for k, v in self.headers.iteritems():
47.1454 + if k.lower() == "content-type":
47.1455 + self.message.set_type(v)
47.1456 + else:
47.1457 + self.message.add_header(k, v)
47.1458 +
47.1459 + self.headers = {}
47.1460 +
47.1461 + def send(self):
47.1462 + try:
47.1463 + import webapi
47.1464 + except ImportError:
47.1465 + webapi = Storage(config=Storage())
47.1466 +
47.1467 + self.prepare_message()
47.1468 + message_text = self.message.as_string()
47.1469 +
47.1470 + if webapi.config.get('smtp_server'):
47.1471 + server = webapi.config.get('smtp_server')
47.1472 + port = webapi.config.get('smtp_port', 0)
47.1473 + username = webapi.config.get('smtp_username')
47.1474 + password = webapi.config.get('smtp_password')
47.1475 + debug_level = webapi.config.get('smtp_debuglevel', None)
47.1476 + starttls = webapi.config.get('smtp_starttls', False)
47.1477 +
47.1478 + import smtplib
47.1479 + smtpserver = smtplib.SMTP(server, port)
47.1480 +
47.1481 + if debug_level:
47.1482 + smtpserver.set_debuglevel(debug_level)
47.1483 +
47.1484 + if starttls:
47.1485 + smtpserver.ehlo()
47.1486 + smtpserver.starttls()
47.1487 + smtpserver.ehlo()
47.1488 +
47.1489 + if username and password:
47.1490 + smtpserver.login(username, password)
47.1491 +
47.1492 + smtpserver.sendmail(self.from_address, self.recipients, message_text)
47.1493 + smtpserver.quit()
47.1494 + elif webapi.config.get('email_engine') == 'aws':
47.1495 + import boto.ses
47.1496 + c = boto.ses.SESConnection(
47.1497 + aws_access_key_id=webapi.config.get('aws_access_key_id'),
47.1498 + aws_secret_access_key=web.api.config.get('aws_secret_access_key'))
47.1499 + c.send_raw_email(self.from_address, message_text, self.from_recipients)
47.1500 + else:
47.1501 + sendmail = webapi.config.get('sendmail_path', '/usr/sbin/sendmail')
47.1502 +
47.1503 + assert not self.from_address.startswith('-'), 'security'
47.1504 + for r in self.recipients:
47.1505 + assert not r.startswith('-'), 'security'
47.1506 +
47.1507 + cmd = [sendmail, '-f', self.from_address] + self.recipients
47.1508 +
47.1509 + if subprocess:
47.1510 + p = subprocess.Popen(cmd, stdin=subprocess.PIPE)
47.1511 + p.stdin.write(message_text)
47.1512 + p.stdin.close()
47.1513 + p.wait()
47.1514 + else:
47.1515 + i, o = os.popen2(cmd)
47.1516 + i.write(message)
47.1517 + i.close()
47.1518 + o.close()
47.1519 + del i, o
47.1520 +
47.1521 + def __repr__(self):
47.1522 + return "<EmailMessage>"
47.1523 +
47.1524 + def __str__(self):
47.1525 + return self.message.as_string()
47.1526 +
47.1527 +if __name__ == "__main__":
47.1528 + import doctest
47.1529 + doctest.testmod()
48.1 --- /dev/null Thu Jan 01 00:00:00 1970 +0000
48.2 +++ b/OpenSecurity/install/web.py-0.37/build/lib/web/webapi.py Mon Dec 02 14:02:05 2013 +0100
48.3 @@ -0,0 +1,525 @@
48.4 +"""
48.5 +Web API (wrapper around WSGI)
48.6 +(from web.py)
48.7 +"""
48.8 +
48.9 +__all__ = [
48.10 + "config",
48.11 + "header", "debug",
48.12 + "input", "data",
48.13 + "setcookie", "cookies",
48.14 + "ctx",
48.15 + "HTTPError",
48.16 +
48.17 + # 200, 201, 202
48.18 + "OK", "Created", "Accepted",
48.19 + "ok", "created", "accepted",
48.20 +
48.21 + # 301, 302, 303, 304, 307
48.22 + "Redirect", "Found", "SeeOther", "NotModified", "TempRedirect",
48.23 + "redirect", "found", "seeother", "notmodified", "tempredirect",
48.24 +
48.25 + # 400, 401, 403, 404, 405, 406, 409, 410, 412, 415
48.26 + "BadRequest", "Unauthorized", "Forbidden", "NotFound", "NoMethod", "NotAcceptable", "Conflict", "Gone", "PreconditionFailed", "UnsupportedMediaType",
48.27 + "badrequest", "unauthorized", "forbidden", "notfound", "nomethod", "notacceptable", "conflict", "gone", "preconditionfailed", "unsupportedmediatype",
48.28 +
48.29 + # 500
48.30 + "InternalError",
48.31 + "internalerror",
48.32 +]
48.33 +
48.34 +import sys, cgi, Cookie, pprint, urlparse, urllib
48.35 +from utils import storage, storify, threadeddict, dictadd, intget, safestr
48.36 +
48.37 +config = storage()
48.38 +config.__doc__ = """
48.39 +A configuration object for various aspects of web.py.
48.40 +
48.41 +`debug`
48.42 + : when True, enables reloading, disabled template caching and sets internalerror to debugerror.
48.43 +"""
48.44 +
48.45 +class HTTPError(Exception):
48.46 + def __init__(self, status, headers={}, data=""):
48.47 + ctx.status = status
48.48 + for k, v in headers.items():
48.49 + header(k, v)
48.50 + self.data = data
48.51 + Exception.__init__(self, status)
48.52 +
48.53 +def _status_code(status, data=None, classname=None, docstring=None):
48.54 + if data is None:
48.55 + data = status.split(" ", 1)[1]
48.56 + classname = status.split(" ", 1)[1].replace(' ', '') # 304 Not Modified -> NotModified
48.57 + docstring = docstring or '`%s` status' % status
48.58 +
48.59 + def __init__(self, data=data, headers={}):
48.60 + HTTPError.__init__(self, status, headers, data)
48.61 +
48.62 + # trick to create class dynamically with dynamic docstring.
48.63 + return type(classname, (HTTPError, object), {
48.64 + '__doc__': docstring,
48.65 + '__init__': __init__
48.66 + })
48.67 +
48.68 +ok = OK = _status_code("200 OK", data="")
48.69 +created = Created = _status_code("201 Created")
48.70 +accepted = Accepted = _status_code("202 Accepted")
48.71 +
48.72 +class Redirect(HTTPError):
48.73 + """A `301 Moved Permanently` redirect."""
48.74 + def __init__(self, url, status='301 Moved Permanently', absolute=False):
48.75 + """
48.76 + Returns a `status` redirect to the new URL.
48.77 + `url` is joined with the base URL so that things like
48.78 + `redirect("about") will work properly.
48.79 + """
48.80 + newloc = urlparse.urljoin(ctx.path, url)
48.81 +
48.82 + if newloc.startswith('/'):
48.83 + if absolute:
48.84 + home = ctx.realhome
48.85 + else:
48.86 + home = ctx.home
48.87 + newloc = home + newloc
48.88 +
48.89 + headers = {
48.90 + 'Content-Type': 'text/html',
48.91 + 'Location': newloc
48.92 + }
48.93 + HTTPError.__init__(self, status, headers, "")
48.94 +
48.95 +redirect = Redirect
48.96 +
48.97 +class Found(Redirect):
48.98 + """A `302 Found` redirect."""
48.99 + def __init__(self, url, absolute=False):
48.100 + Redirect.__init__(self, url, '302 Found', absolute=absolute)
48.101 +
48.102 +found = Found
48.103 +
48.104 +class SeeOther(Redirect):
48.105 + """A `303 See Other` redirect."""
48.106 + def __init__(self, url, absolute=False):
48.107 + Redirect.__init__(self, url, '303 See Other', absolute=absolute)
48.108 +
48.109 +seeother = SeeOther
48.110 +
48.111 +class NotModified(HTTPError):
48.112 + """A `304 Not Modified` status."""
48.113 + def __init__(self):
48.114 + HTTPError.__init__(self, "304 Not Modified")
48.115 +
48.116 +notmodified = NotModified
48.117 +
48.118 +class TempRedirect(Redirect):
48.119 + """A `307 Temporary Redirect` redirect."""
48.120 + def __init__(self, url, absolute=False):
48.121 + Redirect.__init__(self, url, '307 Temporary Redirect', absolute=absolute)
48.122 +
48.123 +tempredirect = TempRedirect
48.124 +
48.125 +class BadRequest(HTTPError):
48.126 + """`400 Bad Request` error."""
48.127 + message = "bad request"
48.128 + def __init__(self, message=None):
48.129 + status = "400 Bad Request"
48.130 + headers = {'Content-Type': 'text/html'}
48.131 + HTTPError.__init__(self, status, headers, message or self.message)
48.132 +
48.133 +badrequest = BadRequest
48.134 +
48.135 +class Unauthorized(HTTPError):
48.136 + """`401 Unauthorized` error."""
48.137 + message = "unauthorized"
48.138 + def __init__(self):
48.139 + status = "401 Unauthorized"
48.140 + headers = {'Content-Type': 'text/html'}
48.141 + HTTPError.__init__(self, status, headers, self.message)
48.142 +
48.143 +unauthorized = Unauthorized
48.144 +
48.145 +class Forbidden(HTTPError):
48.146 + """`403 Forbidden` error."""
48.147 + message = "forbidden"
48.148 + def __init__(self):
48.149 + status = "403 Forbidden"
48.150 + headers = {'Content-Type': 'text/html'}
48.151 + HTTPError.__init__(self, status, headers, self.message)
48.152 +
48.153 +forbidden = Forbidden
48.154 +
48.155 +class _NotFound(HTTPError):
48.156 + """`404 Not Found` error."""
48.157 + message = "not found"
48.158 + def __init__(self, message=None):
48.159 + status = '404 Not Found'
48.160 + headers = {'Content-Type': 'text/html'}
48.161 + HTTPError.__init__(self, status, headers, message or self.message)
48.162 +
48.163 +def NotFound(message=None):
48.164 + """Returns HTTPError with '404 Not Found' error from the active application.
48.165 + """
48.166 + if message:
48.167 + return _NotFound(message)
48.168 + elif ctx.get('app_stack'):
48.169 + return ctx.app_stack[-1].notfound()
48.170 + else:
48.171 + return _NotFound()
48.172 +
48.173 +notfound = NotFound
48.174 +
48.175 +class NoMethod(HTTPError):
48.176 + """A `405 Method Not Allowed` error."""
48.177 + def __init__(self, cls=None):
48.178 + status = '405 Method Not Allowed'
48.179 + headers = {}
48.180 + headers['Content-Type'] = 'text/html'
48.181 +
48.182 + methods = ['GET', 'HEAD', 'POST', 'PUT', 'DELETE']
48.183 + if cls:
48.184 + methods = [method for method in methods if hasattr(cls, method)]
48.185 +
48.186 + headers['Allow'] = ', '.join(methods)
48.187 + data = None
48.188 + HTTPError.__init__(self, status, headers, data)
48.189 +
48.190 +nomethod = NoMethod
48.191 +
48.192 +class NotAcceptable(HTTPError):
48.193 + """`406 Not Acceptable` error."""
48.194 + message = "not acceptable"
48.195 + def __init__(self):
48.196 + status = "406 Not Acceptable"
48.197 + headers = {'Content-Type': 'text/html'}
48.198 + HTTPError.__init__(self, status, headers, self.message)
48.199 +
48.200 +notacceptable = NotAcceptable
48.201 +
48.202 +class Conflict(HTTPError):
48.203 + """`409 Conflict` error."""
48.204 + message = "conflict"
48.205 + def __init__(self):
48.206 + status = "409 Conflict"
48.207 + headers = {'Content-Type': 'text/html'}
48.208 + HTTPError.__init__(self, status, headers, self.message)
48.209 +
48.210 +conflict = Conflict
48.211 +
48.212 +class Gone(HTTPError):
48.213 + """`410 Gone` error."""
48.214 + message = "gone"
48.215 + def __init__(self):
48.216 + status = '410 Gone'
48.217 + headers = {'Content-Type': 'text/html'}
48.218 + HTTPError.__init__(self, status, headers, self.message)
48.219 +
48.220 +gone = Gone
48.221 +
48.222 +class PreconditionFailed(HTTPError):
48.223 + """`412 Precondition Failed` error."""
48.224 + message = "precondition failed"
48.225 + def __init__(self):
48.226 + status = "412 Precondition Failed"
48.227 + headers = {'Content-Type': 'text/html'}
48.228 + HTTPError.__init__(self, status, headers, self.message)
48.229 +
48.230 +preconditionfailed = PreconditionFailed
48.231 +
48.232 +class UnsupportedMediaType(HTTPError):
48.233 + """`415 Unsupported Media Type` error."""
48.234 + message = "unsupported media type"
48.235 + def __init__(self):
48.236 + status = "415 Unsupported Media Type"
48.237 + headers = {'Content-Type': 'text/html'}
48.238 + HTTPError.__init__(self, status, headers, self.message)
48.239 +
48.240 +unsupportedmediatype = UnsupportedMediaType
48.241 +
48.242 +class _InternalError(HTTPError):
48.243 + """500 Internal Server Error`."""
48.244 + message = "internal server error"
48.245 +
48.246 + def __init__(self, message=None):
48.247 + status = '500 Internal Server Error'
48.248 + headers = {'Content-Type': 'text/html'}
48.249 + HTTPError.__init__(self, status, headers, message or self.message)
48.250 +
48.251 +def InternalError(message=None):
48.252 + """Returns HTTPError with '500 internal error' error from the active application.
48.253 + """
48.254 + if message:
48.255 + return _InternalError(message)
48.256 + elif ctx.get('app_stack'):
48.257 + return ctx.app_stack[-1].internalerror()
48.258 + else:
48.259 + return _InternalError()
48.260 +
48.261 +internalerror = InternalError
48.262 +
48.263 +def header(hdr, value, unique=False):
48.264 + """
48.265 + Adds the header `hdr: value` with the response.
48.266 +
48.267 + If `unique` is True and a header with that name already exists,
48.268 + it doesn't add a new one.
48.269 + """
48.270 + hdr, value = safestr(hdr), safestr(value)
48.271 + # protection against HTTP response splitting attack
48.272 + if '\n' in hdr or '\r' in hdr or '\n' in value or '\r' in value:
48.273 + raise ValueError, 'invalid characters in header'
48.274 +
48.275 + if unique is True:
48.276 + for h, v in ctx.headers:
48.277 + if h.lower() == hdr.lower(): return
48.278 +
48.279 + ctx.headers.append((hdr, value))
48.280 +
48.281 +def rawinput(method=None):
48.282 + """Returns storage object with GET or POST arguments.
48.283 + """
48.284 + method = method or "both"
48.285 + from cStringIO import StringIO
48.286 +
48.287 + def dictify(fs):
48.288 + # hack to make web.input work with enctype='text/plain.
48.289 + if fs.list is None:
48.290 + fs.list = []
48.291 +
48.292 + return dict([(k, fs[k]) for k in fs.keys()])
48.293 +
48.294 + e = ctx.env.copy()
48.295 + a = b = {}
48.296 +
48.297 + if method.lower() in ['both', 'post', 'put']:
48.298 + if e['REQUEST_METHOD'] in ['POST', 'PUT']:
48.299 + if e.get('CONTENT_TYPE', '').lower().startswith('multipart/'):
48.300 + # since wsgi.input is directly passed to cgi.FieldStorage,
48.301 + # it can not be called multiple times. Saving the FieldStorage
48.302 + # object in ctx to allow calling web.input multiple times.
48.303 + a = ctx.get('_fieldstorage')
48.304 + if not a:
48.305 + fp = e['wsgi.input']
48.306 + a = cgi.FieldStorage(fp=fp, environ=e, keep_blank_values=1)
48.307 + ctx._fieldstorage = a
48.308 + else:
48.309 + fp = StringIO(data())
48.310 + a = cgi.FieldStorage(fp=fp, environ=e, keep_blank_values=1)
48.311 + a = dictify(a)
48.312 +
48.313 + if method.lower() in ['both', 'get']:
48.314 + e['REQUEST_METHOD'] = 'GET'
48.315 + b = dictify(cgi.FieldStorage(environ=e, keep_blank_values=1))
48.316 +
48.317 + def process_fieldstorage(fs):
48.318 + if isinstance(fs, list):
48.319 + return [process_fieldstorage(x) for x in fs]
48.320 + elif fs.filename is None:
48.321 + return fs.value
48.322 + else:
48.323 + return fs
48.324 +
48.325 + return storage([(k, process_fieldstorage(v)) for k, v in dictadd(b, a).items()])
48.326 +
48.327 +def input(*requireds, **defaults):
48.328 + """
48.329 + Returns a `storage` object with the GET and POST arguments.
48.330 + See `storify` for how `requireds` and `defaults` work.
48.331 + """
48.332 + _method = defaults.pop('_method', 'both')
48.333 + out = rawinput(_method)
48.334 + try:
48.335 + defaults.setdefault('_unicode', True) # force unicode conversion by default.
48.336 + return storify(out, *requireds, **defaults)
48.337 + except KeyError:
48.338 + raise badrequest()
48.339 +
48.340 +def data():
48.341 + """Returns the data sent with the request."""
48.342 + if 'data' not in ctx:
48.343 + cl = intget(ctx.env.get('CONTENT_LENGTH'), 0)
48.344 + ctx.data = ctx.env['wsgi.input'].read(cl)
48.345 + return ctx.data
48.346 +
48.347 +def setcookie(name, value, expires='', domain=None,
48.348 + secure=False, httponly=False, path=None):
48.349 + """Sets a cookie."""
48.350 + morsel = Cookie.Morsel()
48.351 + name, value = safestr(name), safestr(value)
48.352 + morsel.set(name, value, urllib.quote(value))
48.353 + if expires < 0:
48.354 + expires = -1000000000
48.355 + morsel['expires'] = expires
48.356 + morsel['path'] = path or ctx.homepath+'/'
48.357 + if domain:
48.358 + morsel['domain'] = domain
48.359 + if secure:
48.360 + morsel['secure'] = secure
48.361 + value = morsel.OutputString()
48.362 + if httponly:
48.363 + value += '; httponly'
48.364 + header('Set-Cookie', value)
48.365 +
48.366 +def decode_cookie(value):
48.367 + r"""Safely decodes a cookie value to unicode.
48.368 +
48.369 + Tries us-ascii, utf-8 and io8859 encodings, in that order.
48.370 +
48.371 + >>> decode_cookie('')
48.372 + u''
48.373 + >>> decode_cookie('asdf')
48.374 + u'asdf'
48.375 + >>> decode_cookie('foo \xC3\xA9 bar')
48.376 + u'foo \xe9 bar'
48.377 + >>> decode_cookie('foo \xE9 bar')
48.378 + u'foo \xe9 bar'
48.379 + """
48.380 + try:
48.381 + # First try plain ASCII encoding
48.382 + return unicode(value, 'us-ascii')
48.383 + except UnicodeError:
48.384 + # Then try UTF-8, and if that fails, ISO8859
48.385 + try:
48.386 + return unicode(value, 'utf-8')
48.387 + except UnicodeError:
48.388 + return unicode(value, 'iso8859', 'ignore')
48.389 +
48.390 +def parse_cookies(http_cookie):
48.391 + r"""Parse a HTTP_COOKIE header and return dict of cookie names and decoded values.
48.392 +
48.393 + >>> sorted(parse_cookies('').items())
48.394 + []
48.395 + >>> sorted(parse_cookies('a=1').items())
48.396 + [('a', '1')]
48.397 + >>> sorted(parse_cookies('a=1%202').items())
48.398 + [('a', '1 2')]
48.399 + >>> sorted(parse_cookies('a=Z%C3%A9Z').items())
48.400 + [('a', 'Z\xc3\xa9Z')]
48.401 + >>> sorted(parse_cookies('a=1; b=2; c=3').items())
48.402 + [('a', '1'), ('b', '2'), ('c', '3')]
48.403 + >>> sorted(parse_cookies('a=1; b=w("x")|y=z; c=3').items())
48.404 + [('a', '1'), ('b', 'w('), ('c', '3')]
48.405 + >>> sorted(parse_cookies('a=1; b=w(%22x%22)|y=z; c=3').items())
48.406 + [('a', '1'), ('b', 'w("x")|y=z'), ('c', '3')]
48.407 +
48.408 + >>> sorted(parse_cookies('keebler=E=mc2').items())
48.409 + [('keebler', 'E=mc2')]
48.410 + >>> sorted(parse_cookies(r'keebler="E=mc2; L=\"Loves\"; fudge=\012;"').items())
48.411 + [('keebler', 'E=mc2; L="Loves"; fudge=\n;')]
48.412 + """
48.413 + #print "parse_cookies"
48.414 + if '"' in http_cookie:
48.415 + # HTTP_COOKIE has quotes in it, use slow but correct cookie parsing
48.416 + cookie = Cookie.SimpleCookie()
48.417 + try:
48.418 + cookie.load(http_cookie)
48.419 + except Cookie.CookieError:
48.420 + # If HTTP_COOKIE header is malformed, try at least to load the cookies we can by
48.421 + # first splitting on ';' and loading each attr=value pair separately
48.422 + cookie = Cookie.SimpleCookie()
48.423 + for attr_value in http_cookie.split(';'):
48.424 + try:
48.425 + cookie.load(attr_value)
48.426 + except Cookie.CookieError:
48.427 + pass
48.428 + cookies = dict((k, urllib.unquote(v.value)) for k, v in cookie.iteritems())
48.429 + else:
48.430 + # HTTP_COOKIE doesn't have quotes, use fast cookie parsing
48.431 + cookies = {}
48.432 + for key_value in http_cookie.split(';'):
48.433 + key_value = key_value.split('=', 1)
48.434 + if len(key_value) == 2:
48.435 + key, value = key_value
48.436 + cookies[key.strip()] = urllib.unquote(value.strip())
48.437 + return cookies
48.438 +
48.439 +def cookies(*requireds, **defaults):
48.440 + r"""Returns a `storage` object with all the request cookies in it.
48.441 +
48.442 + See `storify` for how `requireds` and `defaults` work.
48.443 +
48.444 + This is forgiving on bad HTTP_COOKIE input, it tries to parse at least
48.445 + the cookies it can.
48.446 +
48.447 + The values are converted to unicode if _unicode=True is passed.
48.448 + """
48.449 + # If _unicode=True is specified, use decode_cookie to convert cookie value to unicode
48.450 + if defaults.get("_unicode") is True:
48.451 + defaults['_unicode'] = decode_cookie
48.452 +
48.453 + # parse cookie string and cache the result for next time.
48.454 + if '_parsed_cookies' not in ctx:
48.455 + http_cookie = ctx.env.get("HTTP_COOKIE", "")
48.456 + ctx._parsed_cookies = parse_cookies(http_cookie)
48.457 +
48.458 + try:
48.459 + return storify(ctx._parsed_cookies, *requireds, **defaults)
48.460 + except KeyError:
48.461 + badrequest()
48.462 + raise StopIteration
48.463 +
48.464 +def debug(*args):
48.465 + """
48.466 + Prints a prettyprinted version of `args` to stderr.
48.467 + """
48.468 + try:
48.469 + out = ctx.environ['wsgi.errors']
48.470 + except:
48.471 + out = sys.stderr
48.472 + for arg in args:
48.473 + print >> out, pprint.pformat(arg)
48.474 + return ''
48.475 +
48.476 +def _debugwrite(x):
48.477 + try:
48.478 + out = ctx.environ['wsgi.errors']
48.479 + except:
48.480 + out = sys.stderr
48.481 + out.write(x)
48.482 +debug.write = _debugwrite
48.483 +
48.484 +ctx = context = threadeddict()
48.485 +
48.486 +ctx.__doc__ = """
48.487 +A `storage` object containing various information about the request:
48.488 +
48.489 +`environ` (aka `env`)
48.490 + : A dictionary containing the standard WSGI environment variables.
48.491 +
48.492 +`host`
48.493 + : The domain (`Host` header) requested by the user.
48.494 +
48.495 +`home`
48.496 + : The base path for the application.
48.497 +
48.498 +`ip`
48.499 + : The IP address of the requester.
48.500 +
48.501 +`method`
48.502 + : The HTTP method used.
48.503 +
48.504 +`path`
48.505 + : The path request.
48.506 +
48.507 +`query`
48.508 + : If there are no query arguments, the empty string. Otherwise, a `?` followed
48.509 + by the query string.
48.510 +
48.511 +`fullpath`
48.512 + : The full path requested, including query arguments (`== path + query`).
48.513 +
48.514 +### Response Data
48.515 +
48.516 +`status` (default: "200 OK")
48.517 + : The status code to be used in the response.
48.518 +
48.519 +`headers`
48.520 + : A list of 2-tuples to be used in the response.
48.521 +
48.522 +`output`
48.523 + : A string to be used as the response.
48.524 +"""
48.525 +
48.526 +if __name__ == "__main__":
48.527 + import doctest
48.528 + doctest.testmod()
48.529 \ No newline at end of file
49.1 --- /dev/null Thu Jan 01 00:00:00 1970 +0000
49.2 +++ b/OpenSecurity/install/web.py-0.37/build/lib/web/webopenid.py Mon Dec 02 14:02:05 2013 +0100
49.3 @@ -0,0 +1,115 @@
49.4 +"""openid.py: an openid library for web.py
49.5 +
49.6 +Notes:
49.7 +
49.8 + - This will create a file called .openid_secret_key in the
49.9 + current directory with your secret key in it. If someone
49.10 + has access to this file they can log in as any user. And
49.11 + if the app can't find this file for any reason (e.g. you
49.12 + moved the app somewhere else) then each currently logged
49.13 + in user will get logged out.
49.14 +
49.15 + - State must be maintained through the entire auth process
49.16 + -- this means that if you have multiple web.py processes
49.17 + serving one set of URLs or if you restart your app often
49.18 + then log ins will fail. You have to replace sessions and
49.19 + store for things to work.
49.20 +
49.21 + - We set cookies starting with "openid_".
49.22 +
49.23 +"""
49.24 +
49.25 +import os
49.26 +import random
49.27 +import hmac
49.28 +import __init__ as web
49.29 +import openid.consumer.consumer
49.30 +import openid.store.memstore
49.31 +
49.32 +sessions = {}
49.33 +store = openid.store.memstore.MemoryStore()
49.34 +
49.35 +def _secret():
49.36 + try:
49.37 + secret = file('.openid_secret_key').read()
49.38 + except IOError:
49.39 + # file doesn't exist
49.40 + secret = os.urandom(20)
49.41 + file('.openid_secret_key', 'w').write(secret)
49.42 + return secret
49.43 +
49.44 +def _hmac(identity_url):
49.45 + return hmac.new(_secret(), identity_url).hexdigest()
49.46 +
49.47 +def _random_session():
49.48 + n = random.random()
49.49 + while n in sessions:
49.50 + n = random.random()
49.51 + n = str(n)
49.52 + return n
49.53 +
49.54 +def status():
49.55 + oid_hash = web.cookies().get('openid_identity_hash', '').split(',', 1)
49.56 + if len(oid_hash) > 1:
49.57 + oid_hash, identity_url = oid_hash
49.58 + if oid_hash == _hmac(identity_url):
49.59 + return identity_url
49.60 + return None
49.61 +
49.62 +def form(openid_loc):
49.63 + oid = status()
49.64 + if oid:
49.65 + return '''
49.66 + <form method="post" action="%s">
49.67 + <img src="http://openid.net/login-bg.gif" alt="OpenID" />
49.68 + <strong>%s</strong>
49.69 + <input type="hidden" name="action" value="logout" />
49.70 + <input type="hidden" name="return_to" value="%s" />
49.71 + <button type="submit">log out</button>
49.72 + </form>''' % (openid_loc, oid, web.ctx.fullpath)
49.73 + else:
49.74 + return '''
49.75 + <form method="post" action="%s">
49.76 + <input type="text" name="openid" value=""
49.77 + style="background: url(http://openid.net/login-bg.gif) no-repeat; padding-left: 18px; background-position: 0 50%%;" />
49.78 + <input type="hidden" name="return_to" value="%s" />
49.79 + <button type="submit">log in</button>
49.80 + </form>''' % (openid_loc, web.ctx.fullpath)
49.81 +
49.82 +def logout():
49.83 + web.setcookie('openid_identity_hash', '', expires=-1)
49.84 +
49.85 +class host:
49.86 + def POST(self):
49.87 + # unlike the usual scheme of things, the POST is actually called
49.88 + # first here
49.89 + i = web.input(return_to='/')
49.90 + if i.get('action') == 'logout':
49.91 + logout()
49.92 + return web.redirect(i.return_to)
49.93 +
49.94 + i = web.input('openid', return_to='/')
49.95 +
49.96 + n = _random_session()
49.97 + sessions[n] = {'webpy_return_to': i.return_to}
49.98 +
49.99 + c = openid.consumer.consumer.Consumer(sessions[n], store)
49.100 + a = c.begin(i.openid)
49.101 + f = a.redirectURL(web.ctx.home, web.ctx.home + web.ctx.fullpath)
49.102 +
49.103 + web.setcookie('openid_session_id', n)
49.104 + return web.redirect(f)
49.105 +
49.106 + def GET(self):
49.107 + n = web.cookies('openid_session_id').openid_session_id
49.108 + web.setcookie('openid_session_id', '', expires=-1)
49.109 + return_to = sessions[n]['webpy_return_to']
49.110 +
49.111 + c = openid.consumer.consumer.Consumer(sessions[n], store)
49.112 + a = c.complete(web.input(), web.ctx.home + web.ctx.fullpath)
49.113 +
49.114 + if a.status.lower() == 'success':
49.115 + web.setcookie('openid_identity_hash', _hmac(a.identity_url) + ',' + a.identity_url)
49.116 +
49.117 + del sessions[n]
49.118 + return web.redirect(return_to)
50.1 --- /dev/null Thu Jan 01 00:00:00 1970 +0000
50.2 +++ b/OpenSecurity/install/web.py-0.37/build/lib/web/wsgi.py Mon Dec 02 14:02:05 2013 +0100
50.3 @@ -0,0 +1,70 @@
50.4 +"""
50.5 +WSGI Utilities
50.6 +(from web.py)
50.7 +"""
50.8 +
50.9 +import os, sys
50.10 +
50.11 +import http
50.12 +import webapi as web
50.13 +from utils import listget
50.14 +from net import validaddr, validip
50.15 +import httpserver
50.16 +
50.17 +def runfcgi(func, addr=('localhost', 8000)):
50.18 + """Runs a WSGI function as a FastCGI server."""
50.19 + import flup.server.fcgi as flups
50.20 + return flups.WSGIServer(func, multiplexed=True, bindAddress=addr, debug=False).run()
50.21 +
50.22 +def runscgi(func, addr=('localhost', 4000)):
50.23 + """Runs a WSGI function as an SCGI server."""
50.24 + import flup.server.scgi as flups
50.25 + return flups.WSGIServer(func, bindAddress=addr, debug=False).run()
50.26 +
50.27 +def runwsgi(func):
50.28 + """
50.29 + Runs a WSGI-compatible `func` using FCGI, SCGI, or a simple web server,
50.30 + as appropriate based on context and `sys.argv`.
50.31 + """
50.32 +
50.33 + if os.environ.has_key('SERVER_SOFTWARE'): # cgi
50.34 + os.environ['FCGI_FORCE_CGI'] = 'Y'
50.35 +
50.36 + if (os.environ.has_key('PHP_FCGI_CHILDREN') #lighttpd fastcgi
50.37 + or os.environ.has_key('SERVER_SOFTWARE')):
50.38 + return runfcgi(func, None)
50.39 +
50.40 + if 'fcgi' in sys.argv or 'fastcgi' in sys.argv:
50.41 + args = sys.argv[1:]
50.42 + if 'fastcgi' in args: args.remove('fastcgi')
50.43 + elif 'fcgi' in args: args.remove('fcgi')
50.44 + if args:
50.45 + return runfcgi(func, validaddr(args[0]))
50.46 + else:
50.47 + return runfcgi(func, None)
50.48 +
50.49 + if 'scgi' in sys.argv:
50.50 + args = sys.argv[1:]
50.51 + args.remove('scgi')
50.52 + if args:
50.53 + return runscgi(func, validaddr(args[0]))
50.54 + else:
50.55 + return runscgi(func)
50.56 +
50.57 + return httpserver.runsimple(func, validip(listget(sys.argv, 1, '')))
50.58 +
50.59 +def _is_dev_mode():
50.60 + # Some embedded python interpreters won't have sys.arv
50.61 + # For details, see https://github.com/webpy/webpy/issues/87
50.62 + argv = getattr(sys, "argv", [])
50.63 +
50.64 + # quick hack to check if the program is running in dev mode.
50.65 + if os.environ.has_key('SERVER_SOFTWARE') \
50.66 + or os.environ.has_key('PHP_FCGI_CHILDREN') \
50.67 + or 'fcgi' in argv or 'fastcgi' in argv \
50.68 + or 'mod_wsgi' in argv:
50.69 + return False
50.70 + return True
50.71 +
50.72 +# When running the builtin-server, enable debug mode if not already set.
50.73 +web.config.setdefault('debug', _is_dev_mode())
51.1 --- /dev/null Thu Jan 01 00:00:00 1970 +0000
51.2 +++ b/OpenSecurity/install/web.py-0.37/build/lib/web/wsgiserver/__init__.py Mon Dec 02 14:02:05 2013 +0100
51.3 @@ -0,0 +1,2219 @@
51.4 +"""A high-speed, production ready, thread pooled, generic HTTP server.
51.5 +
51.6 +Simplest example on how to use this module directly
51.7 +(without using CherryPy's application machinery)::
51.8 +
51.9 + from cherrypy import wsgiserver
51.10 +
51.11 + def my_crazy_app(environ, start_response):
51.12 + status = '200 OK'
51.13 + response_headers = [('Content-type','text/plain')]
51.14 + start_response(status, response_headers)
51.15 + return ['Hello world!']
51.16 +
51.17 + server = wsgiserver.CherryPyWSGIServer(
51.18 + ('0.0.0.0', 8070), my_crazy_app,
51.19 + server_name='www.cherrypy.example')
51.20 + server.start()
51.21 +
51.22 +The CherryPy WSGI server can serve as many WSGI applications
51.23 +as you want in one instance by using a WSGIPathInfoDispatcher::
51.24 +
51.25 + d = WSGIPathInfoDispatcher({'/': my_crazy_app, '/blog': my_blog_app})
51.26 + server = wsgiserver.CherryPyWSGIServer(('0.0.0.0', 80), d)
51.27 +
51.28 +Want SSL support? Just set server.ssl_adapter to an SSLAdapter instance.
51.29 +
51.30 +This won't call the CherryPy engine (application side) at all, only the
51.31 +HTTP server, which is independent from the rest of CherryPy. Don't
51.32 +let the name "CherryPyWSGIServer" throw you; the name merely reflects
51.33 +its origin, not its coupling.
51.34 +
51.35 +For those of you wanting to understand internals of this module, here's the
51.36 +basic call flow. The server's listening thread runs a very tight loop,
51.37 +sticking incoming connections onto a Queue::
51.38 +
51.39 + server = CherryPyWSGIServer(...)
51.40 + server.start()
51.41 + while True:
51.42 + tick()
51.43 + # This blocks until a request comes in:
51.44 + child = socket.accept()
51.45 + conn = HTTPConnection(child, ...)
51.46 + server.requests.put(conn)
51.47 +
51.48 +Worker threads are kept in a pool and poll the Queue, popping off and then
51.49 +handling each connection in turn. Each connection can consist of an arbitrary
51.50 +number of requests and their responses, so we run a nested loop::
51.51 +
51.52 + while True:
51.53 + conn = server.requests.get()
51.54 + conn.communicate()
51.55 + -> while True:
51.56 + req = HTTPRequest(...)
51.57 + req.parse_request()
51.58 + -> # Read the Request-Line, e.g. "GET /page HTTP/1.1"
51.59 + req.rfile.readline()
51.60 + read_headers(req.rfile, req.inheaders)
51.61 + req.respond()
51.62 + -> response = app(...)
51.63 + try:
51.64 + for chunk in response:
51.65 + if chunk:
51.66 + req.write(chunk)
51.67 + finally:
51.68 + if hasattr(response, "close"):
51.69 + response.close()
51.70 + if req.close_connection:
51.71 + return
51.72 +"""
51.73 +
51.74 +CRLF = '\r\n'
51.75 +import os
51.76 +import Queue
51.77 +import re
51.78 +quoted_slash = re.compile("(?i)%2F")
51.79 +import rfc822
51.80 +import socket
51.81 +import sys
51.82 +if 'win' in sys.platform and not hasattr(socket, 'IPPROTO_IPV6'):
51.83 + socket.IPPROTO_IPV6 = 41
51.84 +try:
51.85 + import cStringIO as StringIO
51.86 +except ImportError:
51.87 + import StringIO
51.88 +DEFAULT_BUFFER_SIZE = -1
51.89 +
51.90 +_fileobject_uses_str_type = isinstance(socket._fileobject(None)._rbuf, basestring)
51.91 +
51.92 +import threading
51.93 +import time
51.94 +import traceback
51.95 +def format_exc(limit=None):
51.96 + """Like print_exc() but return a string. Backport for Python 2.3."""
51.97 + try:
51.98 + etype, value, tb = sys.exc_info()
51.99 + return ''.join(traceback.format_exception(etype, value, tb, limit))
51.100 + finally:
51.101 + etype = value = tb = None
51.102 +
51.103 +
51.104 +from urllib import unquote
51.105 +from urlparse import urlparse
51.106 +import warnings
51.107 +
51.108 +import errno
51.109 +
51.110 +def plat_specific_errors(*errnames):
51.111 + """Return error numbers for all errors in errnames on this platform.
51.112 +
51.113 + The 'errno' module contains different global constants depending on
51.114 + the specific platform (OS). This function will return the list of
51.115 + numeric values for a given list of potential names.
51.116 + """
51.117 + errno_names = dir(errno)
51.118 + nums = [getattr(errno, k) for k in errnames if k in errno_names]
51.119 + # de-dupe the list
51.120 + return dict.fromkeys(nums).keys()
51.121 +
51.122 +socket_error_eintr = plat_specific_errors("EINTR", "WSAEINTR")
51.123 +
51.124 +socket_errors_to_ignore = plat_specific_errors(
51.125 + "EPIPE",
51.126 + "EBADF", "WSAEBADF",
51.127 + "ENOTSOCK", "WSAENOTSOCK",
51.128 + "ETIMEDOUT", "WSAETIMEDOUT",
51.129 + "ECONNREFUSED", "WSAECONNREFUSED",
51.130 + "ECONNRESET", "WSAECONNRESET",
51.131 + "ECONNABORTED", "WSAECONNABORTED",
51.132 + "ENETRESET", "WSAENETRESET",
51.133 + "EHOSTDOWN", "EHOSTUNREACH",
51.134 + )
51.135 +socket_errors_to_ignore.append("timed out")
51.136 +socket_errors_to_ignore.append("The read operation timed out")
51.137 +
51.138 +socket_errors_nonblocking = plat_specific_errors(
51.139 + 'EAGAIN', 'EWOULDBLOCK', 'WSAEWOULDBLOCK')
51.140 +
51.141 +comma_separated_headers = ['Accept', 'Accept-Charset', 'Accept-Encoding',
51.142 + 'Accept-Language', 'Accept-Ranges', 'Allow', 'Cache-Control',
51.143 + 'Connection', 'Content-Encoding', 'Content-Language', 'Expect',
51.144 + 'If-Match', 'If-None-Match', 'Pragma', 'Proxy-Authenticate', 'TE',
51.145 + 'Trailer', 'Transfer-Encoding', 'Upgrade', 'Vary', 'Via', 'Warning',
51.146 + 'WWW-Authenticate']
51.147 +
51.148 +
51.149 +import logging
51.150 +if not hasattr(logging, 'statistics'): logging.statistics = {}
51.151 +
51.152 +
51.153 +def read_headers(rfile, hdict=None):
51.154 + """Read headers from the given stream into the given header dict.
51.155 +
51.156 + If hdict is None, a new header dict is created. Returns the populated
51.157 + header dict.
51.158 +
51.159 + Headers which are repeated are folded together using a comma if their
51.160 + specification so dictates.
51.161 +
51.162 + This function raises ValueError when the read bytes violate the HTTP spec.
51.163 + You should probably return "400 Bad Request" if this happens.
51.164 + """
51.165 + if hdict is None:
51.166 + hdict = {}
51.167 +
51.168 + while True:
51.169 + line = rfile.readline()
51.170 + if not line:
51.171 + # No more data--illegal end of headers
51.172 + raise ValueError("Illegal end of headers.")
51.173 +
51.174 + if line == CRLF:
51.175 + # Normal end of headers
51.176 + break
51.177 + if not line.endswith(CRLF):
51.178 + raise ValueError("HTTP requires CRLF terminators")
51.179 +
51.180 + if line[0] in ' \t':
51.181 + # It's a continuation line.
51.182 + v = line.strip()
51.183 + else:
51.184 + try:
51.185 + k, v = line.split(":", 1)
51.186 + except ValueError:
51.187 + raise ValueError("Illegal header line.")
51.188 + # TODO: what about TE and WWW-Authenticate?
51.189 + k = k.strip().title()
51.190 + v = v.strip()
51.191 + hname = k
51.192 +
51.193 + if k in comma_separated_headers:
51.194 + existing = hdict.get(hname)
51.195 + if existing:
51.196 + v = ", ".join((existing, v))
51.197 + hdict[hname] = v
51.198 +
51.199 + return hdict
51.200 +
51.201 +
51.202 +class MaxSizeExceeded(Exception):
51.203 + pass
51.204 +
51.205 +class SizeCheckWrapper(object):
51.206 + """Wraps a file-like object, raising MaxSizeExceeded if too large."""
51.207 +
51.208 + def __init__(self, rfile, maxlen):
51.209 + self.rfile = rfile
51.210 + self.maxlen = maxlen
51.211 + self.bytes_read = 0
51.212 +
51.213 + def _check_length(self):
51.214 + if self.maxlen and self.bytes_read > self.maxlen:
51.215 + raise MaxSizeExceeded()
51.216 +
51.217 + def read(self, size=None):
51.218 + data = self.rfile.read(size)
51.219 + self.bytes_read += len(data)
51.220 + self._check_length()
51.221 + return data
51.222 +
51.223 + def readline(self, size=None):
51.224 + if size is not None:
51.225 + data = self.rfile.readline(size)
51.226 + self.bytes_read += len(data)
51.227 + self._check_length()
51.228 + return data
51.229 +
51.230 + # User didn't specify a size ...
51.231 + # We read the line in chunks to make sure it's not a 100MB line !
51.232 + res = []
51.233 + while True:
51.234 + data = self.rfile.readline(256)
51.235 + self.bytes_read += len(data)
51.236 + self._check_length()
51.237 + res.append(data)
51.238 + # See http://www.cherrypy.org/ticket/421
51.239 + if len(data) < 256 or data[-1:] == "\n":
51.240 + return ''.join(res)
51.241 +
51.242 + def readlines(self, sizehint=0):
51.243 + # Shamelessly stolen from StringIO
51.244 + total = 0
51.245 + lines = []
51.246 + line = self.readline()
51.247 + while line:
51.248 + lines.append(line)
51.249 + total += len(line)
51.250 + if 0 < sizehint <= total:
51.251 + break
51.252 + line = self.readline()
51.253 + return lines
51.254 +
51.255 + def close(self):
51.256 + self.rfile.close()
51.257 +
51.258 + def __iter__(self):
51.259 + return self
51.260 +
51.261 + def next(self):
51.262 + data = self.rfile.next()
51.263 + self.bytes_read += len(data)
51.264 + self._check_length()
51.265 + return data
51.266 +
51.267 +
51.268 +class KnownLengthRFile(object):
51.269 + """Wraps a file-like object, returning an empty string when exhausted."""
51.270 +
51.271 + def __init__(self, rfile, content_length):
51.272 + self.rfile = rfile
51.273 + self.remaining = content_length
51.274 +
51.275 + def read(self, size=None):
51.276 + if self.remaining == 0:
51.277 + return ''
51.278 + if size is None:
51.279 + size = self.remaining
51.280 + else:
51.281 + size = min(size, self.remaining)
51.282 +
51.283 + data = self.rfile.read(size)
51.284 + self.remaining -= len(data)
51.285 + return data
51.286 +
51.287 + def readline(self, size=None):
51.288 + if self.remaining == 0:
51.289 + return ''
51.290 + if size is None:
51.291 + size = self.remaining
51.292 + else:
51.293 + size = min(size, self.remaining)
51.294 +
51.295 + data = self.rfile.readline(size)
51.296 + self.remaining -= len(data)
51.297 + return data
51.298 +
51.299 + def readlines(self, sizehint=0):
51.300 + # Shamelessly stolen from StringIO
51.301 + total = 0
51.302 + lines = []
51.303 + line = self.readline(sizehint)
51.304 + while line:
51.305 + lines.append(line)
51.306 + total += len(line)
51.307 + if 0 < sizehint <= total:
51.308 + break
51.309 + line = self.readline(sizehint)
51.310 + return lines
51.311 +
51.312 + def close(self):
51.313 + self.rfile.close()
51.314 +
51.315 + def __iter__(self):
51.316 + return self
51.317 +
51.318 + def __next__(self):
51.319 + data = next(self.rfile)
51.320 + self.remaining -= len(data)
51.321 + return data
51.322 +
51.323 +
51.324 +class ChunkedRFile(object):
51.325 + """Wraps a file-like object, returning an empty string when exhausted.
51.326 +
51.327 + This class is intended to provide a conforming wsgi.input value for
51.328 + request entities that have been encoded with the 'chunked' transfer
51.329 + encoding.
51.330 + """
51.331 +
51.332 + def __init__(self, rfile, maxlen, bufsize=8192):
51.333 + self.rfile = rfile
51.334 + self.maxlen = maxlen
51.335 + self.bytes_read = 0
51.336 + self.buffer = ''
51.337 + self.bufsize = bufsize
51.338 + self.closed = False
51.339 +
51.340 + def _fetch(self):
51.341 + if self.closed:
51.342 + return
51.343 +
51.344 + line = self.rfile.readline()
51.345 + self.bytes_read += len(line)
51.346 +
51.347 + if self.maxlen and self.bytes_read > self.maxlen:
51.348 + raise MaxSizeExceeded("Request Entity Too Large", self.maxlen)
51.349 +
51.350 + line = line.strip().split(";", 1)
51.351 +
51.352 + try:
51.353 + chunk_size = line.pop(0)
51.354 + chunk_size = int(chunk_size, 16)
51.355 + except ValueError:
51.356 + raise ValueError("Bad chunked transfer size: " + repr(chunk_size))
51.357 +
51.358 + if chunk_size <= 0:
51.359 + self.closed = True
51.360 + return
51.361 +
51.362 +## if line: chunk_extension = line[0]
51.363 +
51.364 + if self.maxlen and self.bytes_read + chunk_size > self.maxlen:
51.365 + raise IOError("Request Entity Too Large")
51.366 +
51.367 + chunk = self.rfile.read(chunk_size)
51.368 + self.bytes_read += len(chunk)
51.369 + self.buffer += chunk
51.370 +
51.371 + crlf = self.rfile.read(2)
51.372 + if crlf != CRLF:
51.373 + raise ValueError(
51.374 + "Bad chunked transfer coding (expected '\\r\\n', "
51.375 + "got " + repr(crlf) + ")")
51.376 +
51.377 + def read(self, size=None):
51.378 + data = ''
51.379 + while True:
51.380 + if size and len(data) >= size:
51.381 + return data
51.382 +
51.383 + if not self.buffer:
51.384 + self._fetch()
51.385 + if not self.buffer:
51.386 + # EOF
51.387 + return data
51.388 +
51.389 + if size:
51.390 + remaining = size - len(data)
51.391 + data += self.buffer[:remaining]
51.392 + self.buffer = self.buffer[remaining:]
51.393 + else:
51.394 + data += self.buffer
51.395 +
51.396 + def readline(self, size=None):
51.397 + data = ''
51.398 + while True:
51.399 + if size and len(data) >= size:
51.400 + return data
51.401 +
51.402 + if not self.buffer:
51.403 + self._fetch()
51.404 + if not self.buffer:
51.405 + # EOF
51.406 + return data
51.407 +
51.408 + newline_pos = self.buffer.find('\n')
51.409 + if size:
51.410 + if newline_pos == -1:
51.411 + remaining = size - len(data)
51.412 + data += self.buffer[:remaining]
51.413 + self.buffer = self.buffer[remaining:]
51.414 + else:
51.415 + remaining = min(size - len(data), newline_pos)
51.416 + data += self.buffer[:remaining]
51.417 + self.buffer = self.buffer[remaining:]
51.418 + else:
51.419 + if newline_pos == -1:
51.420 + data += self.buffer
51.421 + else:
51.422 + data += self.buffer[:newline_pos]
51.423 + self.buffer = self.buffer[newline_pos:]
51.424 +
51.425 + def readlines(self, sizehint=0):
51.426 + # Shamelessly stolen from StringIO
51.427 + total = 0
51.428 + lines = []
51.429 + line = self.readline(sizehint)
51.430 + while line:
51.431 + lines.append(line)
51.432 + total += len(line)
51.433 + if 0 < sizehint <= total:
51.434 + break
51.435 + line = self.readline(sizehint)
51.436 + return lines
51.437 +
51.438 + def read_trailer_lines(self):
51.439 + if not self.closed:
51.440 + raise ValueError(
51.441 + "Cannot read trailers until the request body has been read.")
51.442 +
51.443 + while True:
51.444 + line = self.rfile.readline()
51.445 + if not line:
51.446 + # No more data--illegal end of headers
51.447 + raise ValueError("Illegal end of headers.")
51.448 +
51.449 + self.bytes_read += len(line)
51.450 + if self.maxlen and self.bytes_read > self.maxlen:
51.451 + raise IOError("Request Entity Too Large")
51.452 +
51.453 + if line == CRLF:
51.454 + # Normal end of headers
51.455 + break
51.456 + if not line.endswith(CRLF):
51.457 + raise ValueError("HTTP requires CRLF terminators")
51.458 +
51.459 + yield line
51.460 +
51.461 + def close(self):
51.462 + self.rfile.close()
51.463 +
51.464 + def __iter__(self):
51.465 + # Shamelessly stolen from StringIO
51.466 + total = 0
51.467 + line = self.readline(sizehint)
51.468 + while line:
51.469 + yield line
51.470 + total += len(line)
51.471 + if 0 < sizehint <= total:
51.472 + break
51.473 + line = self.readline(sizehint)
51.474 +
51.475 +
51.476 +class HTTPRequest(object):
51.477 + """An HTTP Request (and response).
51.478 +
51.479 + A single HTTP connection may consist of multiple request/response pairs.
51.480 + """
51.481 +
51.482 + server = None
51.483 + """The HTTPServer object which is receiving this request."""
51.484 +
51.485 + conn = None
51.486 + """The HTTPConnection object on which this request connected."""
51.487 +
51.488 + inheaders = {}
51.489 + """A dict of request headers."""
51.490 +
51.491 + outheaders = []
51.492 + """A list of header tuples to write in the response."""
51.493 +
51.494 + ready = False
51.495 + """When True, the request has been parsed and is ready to begin generating
51.496 + the response. When False, signals the calling Connection that the response
51.497 + should not be generated and the connection should close."""
51.498 +
51.499 + close_connection = False
51.500 + """Signals the calling Connection that the request should close. This does
51.501 + not imply an error! The client and/or server may each request that the
51.502 + connection be closed."""
51.503 +
51.504 + chunked_write = False
51.505 + """If True, output will be encoded with the "chunked" transfer-coding.
51.506 +
51.507 + This value is set automatically inside send_headers."""
51.508 +
51.509 + def __init__(self, server, conn):
51.510 + self.server= server
51.511 + self.conn = conn
51.512 +
51.513 + self.ready = False
51.514 + self.started_request = False
51.515 + self.scheme = "http"
51.516 + if self.server.ssl_adapter is not None:
51.517 + self.scheme = "https"
51.518 + # Use the lowest-common protocol in case read_request_line errors.
51.519 + self.response_protocol = 'HTTP/1.0'
51.520 + self.inheaders = {}
51.521 +
51.522 + self.status = ""
51.523 + self.outheaders = []
51.524 + self.sent_headers = False
51.525 + self.close_connection = self.__class__.close_connection
51.526 + self.chunked_read = False
51.527 + self.chunked_write = self.__class__.chunked_write
51.528 +
51.529 + def parse_request(self):
51.530 + """Parse the next HTTP request start-line and message-headers."""
51.531 + self.rfile = SizeCheckWrapper(self.conn.rfile,
51.532 + self.server.max_request_header_size)
51.533 + try:
51.534 + self.read_request_line()
51.535 + except MaxSizeExceeded:
51.536 + self.simple_response("414 Request-URI Too Long",
51.537 + "The Request-URI sent with the request exceeds the maximum "
51.538 + "allowed bytes.")
51.539 + return
51.540 +
51.541 + try:
51.542 + success = self.read_request_headers()
51.543 + except MaxSizeExceeded:
51.544 + self.simple_response("413 Request Entity Too Large",
51.545 + "The headers sent with the request exceed the maximum "
51.546 + "allowed bytes.")
51.547 + return
51.548 + else:
51.549 + if not success:
51.550 + return
51.551 +
51.552 + self.ready = True
51.553 +
51.554 + def read_request_line(self):
51.555 + # HTTP/1.1 connections are persistent by default. If a client
51.556 + # requests a page, then idles (leaves the connection open),
51.557 + # then rfile.readline() will raise socket.error("timed out").
51.558 + # Note that it does this based on the value given to settimeout(),
51.559 + # and doesn't need the client to request or acknowledge the close
51.560 + # (although your TCP stack might suffer for it: cf Apache's history
51.561 + # with FIN_WAIT_2).
51.562 + request_line = self.rfile.readline()
51.563 +
51.564 + # Set started_request to True so communicate() knows to send 408
51.565 + # from here on out.
51.566 + self.started_request = True
51.567 + if not request_line:
51.568 + # Force self.ready = False so the connection will close.
51.569 + self.ready = False
51.570 + return
51.571 +
51.572 + if request_line == CRLF:
51.573 + # RFC 2616 sec 4.1: "...if the server is reading the protocol
51.574 + # stream at the beginning of a message and receives a CRLF
51.575 + # first, it should ignore the CRLF."
51.576 + # But only ignore one leading line! else we enable a DoS.
51.577 + request_line = self.rfile.readline()
51.578 + if not request_line:
51.579 + self.ready = False
51.580 + return
51.581 +
51.582 + if not request_line.endswith(CRLF):
51.583 + self.simple_response("400 Bad Request", "HTTP requires CRLF terminators")
51.584 + return
51.585 +
51.586 + try:
51.587 + method, uri, req_protocol = request_line.strip().split(" ", 2)
51.588 + rp = int(req_protocol[5]), int(req_protocol[7])
51.589 + except (ValueError, IndexError):
51.590 + self.simple_response("400 Bad Request", "Malformed Request-Line")
51.591 + return
51.592 +
51.593 + self.uri = uri
51.594 + self.method = method
51.595 +
51.596 + # uri may be an abs_path (including "http://host.domain.tld");
51.597 + scheme, authority, path = self.parse_request_uri(uri)
51.598 + if '#' in path:
51.599 + self.simple_response("400 Bad Request",
51.600 + "Illegal #fragment in Request-URI.")
51.601 + return
51.602 +
51.603 + if scheme:
51.604 + self.scheme = scheme
51.605 +
51.606 + qs = ''
51.607 + if '?' in path:
51.608 + path, qs = path.split('?', 1)
51.609 +
51.610 + # Unquote the path+params (e.g. "/this%20path" -> "/this path").
51.611 + # http://www.w3.org/Protocols/rfc2616/rfc2616-sec5.html#sec5.1.2
51.612 + #
51.613 + # But note that "...a URI must be separated into its components
51.614 + # before the escaped characters within those components can be
51.615 + # safely decoded." http://www.ietf.org/rfc/rfc2396.txt, sec 2.4.2
51.616 + # Therefore, "/this%2Fpath" becomes "/this%2Fpath", not "/this/path".
51.617 + try:
51.618 + atoms = [unquote(x) for x in quoted_slash.split(path)]
51.619 + except ValueError, ex:
51.620 + self.simple_response("400 Bad Request", ex.args[0])
51.621 + return
51.622 + path = "%2F".join(atoms)
51.623 + self.path = path
51.624 +
51.625 + # Note that, like wsgiref and most other HTTP servers,
51.626 + # we "% HEX HEX"-unquote the path but not the query string.
51.627 + self.qs = qs
51.628 +
51.629 + # Compare request and server HTTP protocol versions, in case our
51.630 + # server does not support the requested protocol. Limit our output
51.631 + # to min(req, server). We want the following output:
51.632 + # request server actual written supported response
51.633 + # protocol protocol response protocol feature set
51.634 + # a 1.0 1.0 1.0 1.0
51.635 + # b 1.0 1.1 1.1 1.0
51.636 + # c 1.1 1.0 1.0 1.0
51.637 + # d 1.1 1.1 1.1 1.1
51.638 + # Notice that, in (b), the response will be "HTTP/1.1" even though
51.639 + # the client only understands 1.0. RFC 2616 10.5.6 says we should
51.640 + # only return 505 if the _major_ version is different.
51.641 + sp = int(self.server.protocol[5]), int(self.server.protocol[7])
51.642 +
51.643 + if sp[0] != rp[0]:
51.644 + self.simple_response("505 HTTP Version Not Supported")
51.645 + return
51.646 + self.request_protocol = req_protocol
51.647 + self.response_protocol = "HTTP/%s.%s" % min(rp, sp)
51.648 +
51.649 + def read_request_headers(self):
51.650 + """Read self.rfile into self.inheaders. Return success."""
51.651 +
51.652 + # then all the http headers
51.653 + try:
51.654 + read_headers(self.rfile, self.inheaders)
51.655 + except ValueError, ex:
51.656 + self.simple_response("400 Bad Request", ex.args[0])
51.657 + return False
51.658 +
51.659 + mrbs = self.server.max_request_body_size
51.660 + if mrbs and int(self.inheaders.get("Content-Length", 0)) > mrbs:
51.661 + self.simple_response("413 Request Entity Too Large",
51.662 + "The entity sent with the request exceeds the maximum "
51.663 + "allowed bytes.")
51.664 + return False
51.665 +
51.666 + # Persistent connection support
51.667 + if self.response_protocol == "HTTP/1.1":
51.668 + # Both server and client are HTTP/1.1
51.669 + if self.inheaders.get("Connection", "") == "close":
51.670 + self.close_connection = True
51.671 + else:
51.672 + # Either the server or client (or both) are HTTP/1.0
51.673 + if self.inheaders.get("Connection", "") != "Keep-Alive":
51.674 + self.close_connection = True
51.675 +
51.676 + # Transfer-Encoding support
51.677 + te = None
51.678 + if self.response_protocol == "HTTP/1.1":
51.679 + te = self.inheaders.get("Transfer-Encoding")
51.680 + if te:
51.681 + te = [x.strip().lower() for x in te.split(",") if x.strip()]
51.682 +
51.683 + self.chunked_read = False
51.684 +
51.685 + if te:
51.686 + for enc in te:
51.687 + if enc == "chunked":
51.688 + self.chunked_read = True
51.689 + else:
51.690 + # Note that, even if we see "chunked", we must reject
51.691 + # if there is an extension we don't recognize.
51.692 + self.simple_response("501 Unimplemented")
51.693 + self.close_connection = True
51.694 + return False
51.695 +
51.696 + # From PEP 333:
51.697 + # "Servers and gateways that implement HTTP 1.1 must provide
51.698 + # transparent support for HTTP 1.1's "expect/continue" mechanism.
51.699 + # This may be done in any of several ways:
51.700 + # 1. Respond to requests containing an Expect: 100-continue request
51.701 + # with an immediate "100 Continue" response, and proceed normally.
51.702 + # 2. Proceed with the request normally, but provide the application
51.703 + # with a wsgi.input stream that will send the "100 Continue"
51.704 + # response if/when the application first attempts to read from
51.705 + # the input stream. The read request must then remain blocked
51.706 + # until the client responds.
51.707 + # 3. Wait until the client decides that the server does not support
51.708 + # expect/continue, and sends the request body on its own.
51.709 + # (This is suboptimal, and is not recommended.)
51.710 + #
51.711 + # We used to do 3, but are now doing 1. Maybe we'll do 2 someday,
51.712 + # but it seems like it would be a big slowdown for such a rare case.
51.713 + if self.inheaders.get("Expect", "") == "100-continue":
51.714 + # Don't use simple_response here, because it emits headers
51.715 + # we don't want. See http://www.cherrypy.org/ticket/951
51.716 + msg = self.server.protocol + " 100 Continue\r\n\r\n"
51.717 + try:
51.718 + self.conn.wfile.sendall(msg)
51.719 + except socket.error, x:
51.720 + if x.args[0] not in socket_errors_to_ignore:
51.721 + raise
51.722 + return True
51.723 +
51.724 + def parse_request_uri(self, uri):
51.725 + """Parse a Request-URI into (scheme, authority, path).
51.726 +
51.727 + Note that Request-URI's must be one of::
51.728 +
51.729 + Request-URI = "*" | absoluteURI | abs_path | authority
51.730 +
51.731 + Therefore, a Request-URI which starts with a double forward-slash
51.732 + cannot be a "net_path"::
51.733 +
51.734 + net_path = "//" authority [ abs_path ]
51.735 +
51.736 + Instead, it must be interpreted as an "abs_path" with an empty first
51.737 + path segment::
51.738 +
51.739 + abs_path = "/" path_segments
51.740 + path_segments = segment *( "/" segment )
51.741 + segment = *pchar *( ";" param )
51.742 + param = *pchar
51.743 + """
51.744 + if uri == "*":
51.745 + return None, None, uri
51.746 +
51.747 + i = uri.find('://')
51.748 + if i > 0 and '?' not in uri[:i]:
51.749 + # An absoluteURI.
51.750 + # If there's a scheme (and it must be http or https), then:
51.751 + # http_URL = "http:" "//" host [ ":" port ] [ abs_path [ "?" query ]]
51.752 + scheme, remainder = uri[:i].lower(), uri[i + 3:]
51.753 + authority, path = remainder.split("/", 1)
51.754 + return scheme, authority, path
51.755 +
51.756 + if uri.startswith('/'):
51.757 + # An abs_path.
51.758 + return None, None, uri
51.759 + else:
51.760 + # An authority.
51.761 + return None, uri, None
51.762 +
51.763 + def respond(self):
51.764 + """Call the gateway and write its iterable output."""
51.765 + mrbs = self.server.max_request_body_size
51.766 + if self.chunked_read:
51.767 + self.rfile = ChunkedRFile(self.conn.rfile, mrbs)
51.768 + else:
51.769 + cl = int(self.inheaders.get("Content-Length", 0))
51.770 + if mrbs and mrbs < cl:
51.771 + if not self.sent_headers:
51.772 + self.simple_response("413 Request Entity Too Large",
51.773 + "The entity sent with the request exceeds the maximum "
51.774 + "allowed bytes.")
51.775 + return
51.776 + self.rfile = KnownLengthRFile(self.conn.rfile, cl)
51.777 +
51.778 + self.server.gateway(self).respond()
51.779 +
51.780 + if (self.ready and not self.sent_headers):
51.781 + self.sent_headers = True
51.782 + self.send_headers()
51.783 + if self.chunked_write:
51.784 + self.conn.wfile.sendall("0\r\n\r\n")
51.785 +
51.786 + def simple_response(self, status, msg=""):
51.787 + """Write a simple response back to the client."""
51.788 + status = str(status)
51.789 + buf = [self.server.protocol + " " +
51.790 + status + CRLF,
51.791 + "Content-Length: %s\r\n" % len(msg),
51.792 + "Content-Type: text/plain\r\n"]
51.793 +
51.794 + if status[:3] in ("413", "414"):
51.795 + # Request Entity Too Large / Request-URI Too Long
51.796 + self.close_connection = True
51.797 + if self.response_protocol == 'HTTP/1.1':
51.798 + # This will not be true for 414, since read_request_line
51.799 + # usually raises 414 before reading the whole line, and we
51.800 + # therefore cannot know the proper response_protocol.
51.801 + buf.append("Connection: close\r\n")
51.802 + else:
51.803 + # HTTP/1.0 had no 413/414 status nor Connection header.
51.804 + # Emit 400 instead and trust the message body is enough.
51.805 + status = "400 Bad Request"
51.806 +
51.807 + buf.append(CRLF)
51.808 + if msg:
51.809 + if isinstance(msg, unicode):
51.810 + msg = msg.encode("ISO-8859-1")
51.811 + buf.append(msg)
51.812 +
51.813 + try:
51.814 + self.conn.wfile.sendall("".join(buf))
51.815 + except socket.error, x:
51.816 + if x.args[0] not in socket_errors_to_ignore:
51.817 + raise
51.818 +
51.819 + def write(self, chunk):
51.820 + """Write unbuffered data to the client."""
51.821 + if self.chunked_write and chunk:
51.822 + buf = [hex(len(chunk))[2:], CRLF, chunk, CRLF]
51.823 + self.conn.wfile.sendall("".join(buf))
51.824 + else:
51.825 + self.conn.wfile.sendall(chunk)
51.826 +
51.827 + def send_headers(self):
51.828 + """Assert, process, and send the HTTP response message-headers.
51.829 +
51.830 + You must set self.status, and self.outheaders before calling this.
51.831 + """
51.832 + hkeys = [key.lower() for key, value in self.outheaders]
51.833 + status = int(self.status[:3])
51.834 +
51.835 + if status == 413:
51.836 + # Request Entity Too Large. Close conn to avoid garbage.
51.837 + self.close_connection = True
51.838 + elif "content-length" not in hkeys:
51.839 + # "All 1xx (informational), 204 (no content),
51.840 + # and 304 (not modified) responses MUST NOT
51.841 + # include a message-body." So no point chunking.
51.842 + if status < 200 or status in (204, 205, 304):
51.843 + pass
51.844 + else:
51.845 + if (self.response_protocol == 'HTTP/1.1'
51.846 + and self.method != 'HEAD'):
51.847 + # Use the chunked transfer-coding
51.848 + self.chunked_write = True
51.849 + self.outheaders.append(("Transfer-Encoding", "chunked"))
51.850 + else:
51.851 + # Closing the conn is the only way to determine len.
51.852 + self.close_connection = True
51.853 +
51.854 + if "connection" not in hkeys:
51.855 + if self.response_protocol == 'HTTP/1.1':
51.856 + # Both server and client are HTTP/1.1 or better
51.857 + if self.close_connection:
51.858 + self.outheaders.append(("Connection", "close"))
51.859 + else:
51.860 + # Server and/or client are HTTP/1.0
51.861 + if not self.close_connection:
51.862 + self.outheaders.append(("Connection", "Keep-Alive"))
51.863 +
51.864 + if (not self.close_connection) and (not self.chunked_read):
51.865 + # Read any remaining request body data on the socket.
51.866 + # "If an origin server receives a request that does not include an
51.867 + # Expect request-header field with the "100-continue" expectation,
51.868 + # the request includes a request body, and the server responds
51.869 + # with a final status code before reading the entire request body
51.870 + # from the transport connection, then the server SHOULD NOT close
51.871 + # the transport connection until it has read the entire request,
51.872 + # or until the client closes the connection. Otherwise, the client
51.873 + # might not reliably receive the response message. However, this
51.874 + # requirement is not be construed as preventing a server from
51.875 + # defending itself against denial-of-service attacks, or from
51.876 + # badly broken client implementations."
51.877 + remaining = getattr(self.rfile, 'remaining', 0)
51.878 + if remaining > 0:
51.879 + self.rfile.read(remaining)
51.880 +
51.881 + if "date" not in hkeys:
51.882 + self.outheaders.append(("Date", rfc822.formatdate()))
51.883 +
51.884 + if "server" not in hkeys:
51.885 + self.outheaders.append(("Server", self.server.server_name))
51.886 +
51.887 + buf = [self.server.protocol + " " + self.status + CRLF]
51.888 + for k, v in self.outheaders:
51.889 + buf.append(k + ": " + v + CRLF)
51.890 + buf.append(CRLF)
51.891 + self.conn.wfile.sendall("".join(buf))
51.892 +
51.893 +
51.894 +class NoSSLError(Exception):
51.895 + """Exception raised when a client speaks HTTP to an HTTPS socket."""
51.896 + pass
51.897 +
51.898 +
51.899 +class FatalSSLAlert(Exception):
51.900 + """Exception raised when the SSL implementation signals a fatal alert."""
51.901 + pass
51.902 +
51.903 +
51.904 +class CP_fileobject(socket._fileobject):
51.905 + """Faux file object attached to a socket object."""
51.906 +
51.907 + def __init__(self, *args, **kwargs):
51.908 + self.bytes_read = 0
51.909 + self.bytes_written = 0
51.910 + socket._fileobject.__init__(self, *args, **kwargs)
51.911 +
51.912 + def sendall(self, data):
51.913 + """Sendall for non-blocking sockets."""
51.914 + while data:
51.915 + try:
51.916 + bytes_sent = self.send(data)
51.917 + data = data[bytes_sent:]
51.918 + except socket.error, e:
51.919 + if e.args[0] not in socket_errors_nonblocking:
51.920 + raise
51.921 +
51.922 + def send(self, data):
51.923 + bytes_sent = self._sock.send(data)
51.924 + self.bytes_written += bytes_sent
51.925 + return bytes_sent
51.926 +
51.927 + def flush(self):
51.928 + if self._wbuf:
51.929 + buffer = "".join(self._wbuf)
51.930 + self._wbuf = []
51.931 + self.sendall(buffer)
51.932 +
51.933 + def recv(self, size):
51.934 + while True:
51.935 + try:
51.936 + data = self._sock.recv(size)
51.937 + self.bytes_read += len(data)
51.938 + return data
51.939 + except socket.error, e:
51.940 + if (e.args[0] not in socket_errors_nonblocking
51.941 + and e.args[0] not in socket_error_eintr):
51.942 + raise
51.943 +
51.944 + if not _fileobject_uses_str_type:
51.945 + def read(self, size=-1):
51.946 + # Use max, disallow tiny reads in a loop as they are very inefficient.
51.947 + # We never leave read() with any leftover data from a new recv() call
51.948 + # in our internal buffer.
51.949 + rbufsize = max(self._rbufsize, self.default_bufsize)
51.950 + # Our use of StringIO rather than lists of string objects returned by
51.951 + # recv() minimizes memory usage and fragmentation that occurs when
51.952 + # rbufsize is large compared to the typical return value of recv().
51.953 + buf = self._rbuf
51.954 + buf.seek(0, 2) # seek end
51.955 + if size < 0:
51.956 + # Read until EOF
51.957 + self._rbuf = StringIO.StringIO() # reset _rbuf. we consume it via buf.
51.958 + while True:
51.959 + data = self.recv(rbufsize)
51.960 + if not data:
51.961 + break
51.962 + buf.write(data)
51.963 + return buf.getvalue()
51.964 + else:
51.965 + # Read until size bytes or EOF seen, whichever comes first
51.966 + buf_len = buf.tell()
51.967 + if buf_len >= size:
51.968 + # Already have size bytes in our buffer? Extract and return.
51.969 + buf.seek(0)
51.970 + rv = buf.read(size)
51.971 + self._rbuf = StringIO.StringIO()
51.972 + self._rbuf.write(buf.read())
51.973 + return rv
51.974 +
51.975 + self._rbuf = StringIO.StringIO() # reset _rbuf. we consume it via buf.
51.976 + while True:
51.977 + left = size - buf_len
51.978 + # recv() will malloc the amount of memory given as its
51.979 + # parameter even though it often returns much less data
51.980 + # than that. The returned data string is short lived
51.981 + # as we copy it into a StringIO and free it. This avoids
51.982 + # fragmentation issues on many platforms.
51.983 + data = self.recv(left)
51.984 + if not data:
51.985 + break
51.986 + n = len(data)
51.987 + if n == size and not buf_len:
51.988 + # Shortcut. Avoid buffer data copies when:
51.989 + # - We have no data in our buffer.
51.990 + # AND
51.991 + # - Our call to recv returned exactly the
51.992 + # number of bytes we were asked to read.
51.993 + return data
51.994 + if n == left:
51.995 + buf.write(data)
51.996 + del data # explicit free
51.997 + break
51.998 + assert n <= left, "recv(%d) returned %d bytes" % (left, n)
51.999 + buf.write(data)
51.1000 + buf_len += n
51.1001 + del data # explicit free
51.1002 + #assert buf_len == buf.tell()
51.1003 + return buf.getvalue()
51.1004 +
51.1005 + def readline(self, size=-1):
51.1006 + buf = self._rbuf
51.1007 + buf.seek(0, 2) # seek end
51.1008 + if buf.tell() > 0:
51.1009 + # check if we already have it in our buffer
51.1010 + buf.seek(0)
51.1011 + bline = buf.readline(size)
51.1012 + if bline.endswith('\n') or len(bline) == size:
51.1013 + self._rbuf = StringIO.StringIO()
51.1014 + self._rbuf.write(buf.read())
51.1015 + return bline
51.1016 + del bline
51.1017 + if size < 0:
51.1018 + # Read until \n or EOF, whichever comes first
51.1019 + if self._rbufsize <= 1:
51.1020 + # Speed up unbuffered case
51.1021 + buf.seek(0)
51.1022 + buffers = [buf.read()]
51.1023 + self._rbuf = StringIO.StringIO() # reset _rbuf. we consume it via buf.
51.1024 + data = None
51.1025 + recv = self.recv
51.1026 + while data != "\n":
51.1027 + data = recv(1)
51.1028 + if not data:
51.1029 + break
51.1030 + buffers.append(data)
51.1031 + return "".join(buffers)
51.1032 +
51.1033 + buf.seek(0, 2) # seek end
51.1034 + self._rbuf = StringIO.StringIO() # reset _rbuf. we consume it via buf.
51.1035 + while True:
51.1036 + data = self.recv(self._rbufsize)
51.1037 + if not data:
51.1038 + break
51.1039 + nl = data.find('\n')
51.1040 + if nl >= 0:
51.1041 + nl += 1
51.1042 + buf.write(data[:nl])
51.1043 + self._rbuf.write(data[nl:])
51.1044 + del data
51.1045 + break
51.1046 + buf.write(data)
51.1047 + return buf.getvalue()
51.1048 + else:
51.1049 + # Read until size bytes or \n or EOF seen, whichever comes first
51.1050 + buf.seek(0, 2) # seek end
51.1051 + buf_len = buf.tell()
51.1052 + if buf_len >= size:
51.1053 + buf.seek(0)
51.1054 + rv = buf.read(size)
51.1055 + self._rbuf = StringIO.StringIO()
51.1056 + self._rbuf.write(buf.read())
51.1057 + return rv
51.1058 + self._rbuf = StringIO.StringIO() # reset _rbuf. we consume it via buf.
51.1059 + while True:
51.1060 + data = self.recv(self._rbufsize)
51.1061 + if not data:
51.1062 + break
51.1063 + left = size - buf_len
51.1064 + # did we just receive a newline?
51.1065 + nl = data.find('\n', 0, left)
51.1066 + if nl >= 0:
51.1067 + nl += 1
51.1068 + # save the excess data to _rbuf
51.1069 + self._rbuf.write(data[nl:])
51.1070 + if buf_len:
51.1071 + buf.write(data[:nl])
51.1072 + break
51.1073 + else:
51.1074 + # Shortcut. Avoid data copy through buf when returning
51.1075 + # a substring of our first recv().
51.1076 + return data[:nl]
51.1077 + n = len(data)
51.1078 + if n == size and not buf_len:
51.1079 + # Shortcut. Avoid data copy through buf when
51.1080 + # returning exactly all of our first recv().
51.1081 + return data
51.1082 + if n >= left:
51.1083 + buf.write(data[:left])
51.1084 + self._rbuf.write(data[left:])
51.1085 + break
51.1086 + buf.write(data)
51.1087 + buf_len += n
51.1088 + #assert buf_len == buf.tell()
51.1089 + return buf.getvalue()
51.1090 + else:
51.1091 + def read(self, size=-1):
51.1092 + if size < 0:
51.1093 + # Read until EOF
51.1094 + buffers = [self._rbuf]
51.1095 + self._rbuf = ""
51.1096 + if self._rbufsize <= 1:
51.1097 + recv_size = self.default_bufsize
51.1098 + else:
51.1099 + recv_size = self._rbufsize
51.1100 +
51.1101 + while True:
51.1102 + data = self.recv(recv_size)
51.1103 + if not data:
51.1104 + break
51.1105 + buffers.append(data)
51.1106 + return "".join(buffers)
51.1107 + else:
51.1108 + # Read until size bytes or EOF seen, whichever comes first
51.1109 + data = self._rbuf
51.1110 + buf_len = len(data)
51.1111 + if buf_len >= size:
51.1112 + self._rbuf = data[size:]
51.1113 + return data[:size]
51.1114 + buffers = []
51.1115 + if data:
51.1116 + buffers.append(data)
51.1117 + self._rbuf = ""
51.1118 + while True:
51.1119 + left = size - buf_len
51.1120 + recv_size = max(self._rbufsize, left)
51.1121 + data = self.recv(recv_size)
51.1122 + if not data:
51.1123 + break
51.1124 + buffers.append(data)
51.1125 + n = len(data)
51.1126 + if n >= left:
51.1127 + self._rbuf = data[left:]
51.1128 + buffers[-1] = data[:left]
51.1129 + break
51.1130 + buf_len += n
51.1131 + return "".join(buffers)
51.1132 +
51.1133 + def readline(self, size=-1):
51.1134 + data = self._rbuf
51.1135 + if size < 0:
51.1136 + # Read until \n or EOF, whichever comes first
51.1137 + if self._rbufsize <= 1:
51.1138 + # Speed up unbuffered case
51.1139 + assert data == ""
51.1140 + buffers = []
51.1141 + while data != "\n":
51.1142 + data = self.recv(1)
51.1143 + if not data:
51.1144 + break
51.1145 + buffers.append(data)
51.1146 + return "".join(buffers)
51.1147 + nl = data.find('\n')
51.1148 + if nl >= 0:
51.1149 + nl += 1
51.1150 + self._rbuf = data[nl:]
51.1151 + return data[:nl]
51.1152 + buffers = []
51.1153 + if data:
51.1154 + buffers.append(data)
51.1155 + self._rbuf = ""
51.1156 + while True:
51.1157 + data = self.recv(self._rbufsize)
51.1158 + if not data:
51.1159 + break
51.1160 + buffers.append(data)
51.1161 + nl = data.find('\n')
51.1162 + if nl >= 0:
51.1163 + nl += 1
51.1164 + self._rbuf = data[nl:]
51.1165 + buffers[-1] = data[:nl]
51.1166 + break
51.1167 + return "".join(buffers)
51.1168 + else:
51.1169 + # Read until size bytes or \n or EOF seen, whichever comes first
51.1170 + nl = data.find('\n', 0, size)
51.1171 + if nl >= 0:
51.1172 + nl += 1
51.1173 + self._rbuf = data[nl:]
51.1174 + return data[:nl]
51.1175 + buf_len = len(data)
51.1176 + if buf_len >= size:
51.1177 + self._rbuf = data[size:]
51.1178 + return data[:size]
51.1179 + buffers = []
51.1180 + if data:
51.1181 + buffers.append(data)
51.1182 + self._rbuf = ""
51.1183 + while True:
51.1184 + data = self.recv(self._rbufsize)
51.1185 + if not data:
51.1186 + break
51.1187 + buffers.append(data)
51.1188 + left = size - buf_len
51.1189 + nl = data.find('\n', 0, left)
51.1190 + if nl >= 0:
51.1191 + nl += 1
51.1192 + self._rbuf = data[nl:]
51.1193 + buffers[-1] = data[:nl]
51.1194 + break
51.1195 + n = len(data)
51.1196 + if n >= left:
51.1197 + self._rbuf = data[left:]
51.1198 + buffers[-1] = data[:left]
51.1199 + break
51.1200 + buf_len += n
51.1201 + return "".join(buffers)
51.1202 +
51.1203 +
51.1204 +class HTTPConnection(object):
51.1205 + """An HTTP connection (active socket).
51.1206 +
51.1207 + server: the Server object which received this connection.
51.1208 + socket: the raw socket object (usually TCP) for this connection.
51.1209 + makefile: a fileobject class for reading from the socket.
51.1210 + """
51.1211 +
51.1212 + remote_addr = None
51.1213 + remote_port = None
51.1214 + ssl_env = None
51.1215 + rbufsize = DEFAULT_BUFFER_SIZE
51.1216 + wbufsize = DEFAULT_BUFFER_SIZE
51.1217 + RequestHandlerClass = HTTPRequest
51.1218 +
51.1219 + def __init__(self, server, sock, makefile=CP_fileobject):
51.1220 + self.server = server
51.1221 + self.socket = sock
51.1222 + self.rfile = makefile(sock, "rb", self.rbufsize)
51.1223 + self.wfile = makefile(sock, "wb", self.wbufsize)
51.1224 + self.requests_seen = 0
51.1225 +
51.1226 + def communicate(self):
51.1227 + """Read each request and respond appropriately."""
51.1228 + request_seen = False
51.1229 + try:
51.1230 + while True:
51.1231 + # (re)set req to None so that if something goes wrong in
51.1232 + # the RequestHandlerClass constructor, the error doesn't
51.1233 + # get written to the previous request.
51.1234 + req = None
51.1235 + req = self.RequestHandlerClass(self.server, self)
51.1236 +
51.1237 + # This order of operations should guarantee correct pipelining.
51.1238 + req.parse_request()
51.1239 + if self.server.stats['Enabled']:
51.1240 + self.requests_seen += 1
51.1241 + if not req.ready:
51.1242 + # Something went wrong in the parsing (and the server has
51.1243 + # probably already made a simple_response). Return and
51.1244 + # let the conn close.
51.1245 + return
51.1246 +
51.1247 + request_seen = True
51.1248 + req.respond()
51.1249 + if req.close_connection:
51.1250 + return
51.1251 + except socket.error, e:
51.1252 + errnum = e.args[0]
51.1253 + # sadly SSL sockets return a different (longer) time out string
51.1254 + if errnum == 'timed out' or errnum == 'The read operation timed out':
51.1255 + # Don't error if we're between requests; only error
51.1256 + # if 1) no request has been started at all, or 2) we're
51.1257 + # in the middle of a request.
51.1258 + # See http://www.cherrypy.org/ticket/853
51.1259 + if (not request_seen) or (req and req.started_request):
51.1260 + # Don't bother writing the 408 if the response
51.1261 + # has already started being written.
51.1262 + if req and not req.sent_headers:
51.1263 + try:
51.1264 + req.simple_response("408 Request Timeout")
51.1265 + except FatalSSLAlert:
51.1266 + # Close the connection.
51.1267 + return
51.1268 + elif errnum not in socket_errors_to_ignore:
51.1269 + if req and not req.sent_headers:
51.1270 + try:
51.1271 + req.simple_response("500 Internal Server Error",
51.1272 + format_exc())
51.1273 + except FatalSSLAlert:
51.1274 + # Close the connection.
51.1275 + return
51.1276 + return
51.1277 + except (KeyboardInterrupt, SystemExit):
51.1278 + raise
51.1279 + except FatalSSLAlert:
51.1280 + # Close the connection.
51.1281 + return
51.1282 + except NoSSLError:
51.1283 + if req and not req.sent_headers:
51.1284 + # Unwrap our wfile
51.1285 + self.wfile = CP_fileobject(self.socket._sock, "wb", self.wbufsize)
51.1286 + req.simple_response("400 Bad Request",
51.1287 + "The client sent a plain HTTP request, but "
51.1288 + "this server only speaks HTTPS on this port.")
51.1289 + self.linger = True
51.1290 + except Exception:
51.1291 + if req and not req.sent_headers:
51.1292 + try:
51.1293 + req.simple_response("500 Internal Server Error", format_exc())
51.1294 + except FatalSSLAlert:
51.1295 + # Close the connection.
51.1296 + return
51.1297 +
51.1298 + linger = False
51.1299 +
51.1300 + def close(self):
51.1301 + """Close the socket underlying this connection."""
51.1302 + self.rfile.close()
51.1303 +
51.1304 + if not self.linger:
51.1305 + # Python's socket module does NOT call close on the kernel socket
51.1306 + # when you call socket.close(). We do so manually here because we
51.1307 + # want this server to send a FIN TCP segment immediately. Note this
51.1308 + # must be called *before* calling socket.close(), because the latter
51.1309 + # drops its reference to the kernel socket.
51.1310 + if hasattr(self.socket, '_sock'):
51.1311 + self.socket._sock.close()
51.1312 + self.socket.close()
51.1313 + else:
51.1314 + # On the other hand, sometimes we want to hang around for a bit
51.1315 + # to make sure the client has a chance to read our entire
51.1316 + # response. Skipping the close() calls here delays the FIN
51.1317 + # packet until the socket object is garbage-collected later.
51.1318 + # Someday, perhaps, we'll do the full lingering_close that
51.1319 + # Apache does, but not today.
51.1320 + pass
51.1321 +
51.1322 +
51.1323 +_SHUTDOWNREQUEST = None
51.1324 +
51.1325 +class WorkerThread(threading.Thread):
51.1326 + """Thread which continuously polls a Queue for Connection objects.
51.1327 +
51.1328 + Due to the timing issues of polling a Queue, a WorkerThread does not
51.1329 + check its own 'ready' flag after it has started. To stop the thread,
51.1330 + it is necessary to stick a _SHUTDOWNREQUEST object onto the Queue
51.1331 + (one for each running WorkerThread).
51.1332 + """
51.1333 +
51.1334 + conn = None
51.1335 + """The current connection pulled off the Queue, or None."""
51.1336 +
51.1337 + server = None
51.1338 + """The HTTP Server which spawned this thread, and which owns the
51.1339 + Queue and is placing active connections into it."""
51.1340 +
51.1341 + ready = False
51.1342 + """A simple flag for the calling server to know when this thread
51.1343 + has begun polling the Queue."""
51.1344 +
51.1345 +
51.1346 + def __init__(self, server):
51.1347 + self.ready = False
51.1348 + self.server = server
51.1349 +
51.1350 + self.requests_seen = 0
51.1351 + self.bytes_read = 0
51.1352 + self.bytes_written = 0
51.1353 + self.start_time = None
51.1354 + self.work_time = 0
51.1355 + self.stats = {
51.1356 + 'Requests': lambda s: self.requests_seen + ((self.start_time is None) and 0 or self.conn.requests_seen),
51.1357 + 'Bytes Read': lambda s: self.bytes_read + ((self.start_time is None) and 0 or self.conn.rfile.bytes_read),
51.1358 + 'Bytes Written': lambda s: self.bytes_written + ((self.start_time is None) and 0 or self.conn.wfile.bytes_written),
51.1359 + 'Work Time': lambda s: self.work_time + ((self.start_time is None) and 0 or time.time() - self.start_time),
51.1360 + 'Read Throughput': lambda s: s['Bytes Read'](s) / (s['Work Time'](s) or 1e-6),
51.1361 + 'Write Throughput': lambda s: s['Bytes Written'](s) / (s['Work Time'](s) or 1e-6),
51.1362 + }
51.1363 + threading.Thread.__init__(self)
51.1364 +
51.1365 + def run(self):
51.1366 + self.server.stats['Worker Threads'][self.getName()] = self.stats
51.1367 + try:
51.1368 + self.ready = True
51.1369 + while True:
51.1370 + conn = self.server.requests.get()
51.1371 + if conn is _SHUTDOWNREQUEST:
51.1372 + return
51.1373 +
51.1374 + self.conn = conn
51.1375 + if self.server.stats['Enabled']:
51.1376 + self.start_time = time.time()
51.1377 + try:
51.1378 + conn.communicate()
51.1379 + finally:
51.1380 + conn.close()
51.1381 + if self.server.stats['Enabled']:
51.1382 + self.requests_seen += self.conn.requests_seen
51.1383 + self.bytes_read += self.conn.rfile.bytes_read
51.1384 + self.bytes_written += self.conn.wfile.bytes_written
51.1385 + self.work_time += time.time() - self.start_time
51.1386 + self.start_time = None
51.1387 + self.conn = None
51.1388 + except (KeyboardInterrupt, SystemExit), exc:
51.1389 + self.server.interrupt = exc
51.1390 +
51.1391 +
51.1392 +class ThreadPool(object):
51.1393 + """A Request Queue for the CherryPyWSGIServer which pools threads.
51.1394 +
51.1395 + ThreadPool objects must provide min, get(), put(obj), start()
51.1396 + and stop(timeout) attributes.
51.1397 + """
51.1398 +
51.1399 + def __init__(self, server, min=10, max=-1):
51.1400 + self.server = server
51.1401 + self.min = min
51.1402 + self.max = max
51.1403 + self._threads = []
51.1404 + self._queue = Queue.Queue()
51.1405 + self.get = self._queue.get
51.1406 +
51.1407 + def start(self):
51.1408 + """Start the pool of threads."""
51.1409 + for i in range(self.min):
51.1410 + self._threads.append(WorkerThread(self.server))
51.1411 + for worker in self._threads:
51.1412 + worker.setName("CP Server " + worker.getName())
51.1413 + worker.start()
51.1414 + for worker in self._threads:
51.1415 + while not worker.ready:
51.1416 + time.sleep(.1)
51.1417 +
51.1418 + def _get_idle(self):
51.1419 + """Number of worker threads which are idle. Read-only."""
51.1420 + return len([t for t in self._threads if t.conn is None])
51.1421 + idle = property(_get_idle, doc=_get_idle.__doc__)
51.1422 +
51.1423 + def put(self, obj):
51.1424 + self._queue.put(obj)
51.1425 + if obj is _SHUTDOWNREQUEST:
51.1426 + return
51.1427 +
51.1428 + def grow(self, amount):
51.1429 + """Spawn new worker threads (not above self.max)."""
51.1430 + for i in range(amount):
51.1431 + if self.max > 0 and len(self._threads) >= self.max:
51.1432 + break
51.1433 + worker = WorkerThread(self.server)
51.1434 + worker.setName("CP Server " + worker.getName())
51.1435 + self._threads.append(worker)
51.1436 + worker.start()
51.1437 +
51.1438 + def shrink(self, amount):
51.1439 + """Kill off worker threads (not below self.min)."""
51.1440 + # Grow/shrink the pool if necessary.
51.1441 + # Remove any dead threads from our list
51.1442 + for t in self._threads:
51.1443 + if not t.isAlive():
51.1444 + self._threads.remove(t)
51.1445 + amount -= 1
51.1446 +
51.1447 + if amount > 0:
51.1448 + for i in range(min(amount, len(self._threads) - self.min)):
51.1449 + # Put a number of shutdown requests on the queue equal
51.1450 + # to 'amount'. Once each of those is processed by a worker,
51.1451 + # that worker will terminate and be culled from our list
51.1452 + # in self.put.
51.1453 + self._queue.put(_SHUTDOWNREQUEST)
51.1454 +
51.1455 + def stop(self, timeout=5):
51.1456 + # Must shut down threads here so the code that calls
51.1457 + # this method can know when all threads are stopped.
51.1458 + for worker in self._threads:
51.1459 + self._queue.put(_SHUTDOWNREQUEST)
51.1460 +
51.1461 + # Don't join currentThread (when stop is called inside a request).
51.1462 + current = threading.currentThread()
51.1463 + if timeout and timeout >= 0:
51.1464 + endtime = time.time() + timeout
51.1465 + while self._threads:
51.1466 + worker = self._threads.pop()
51.1467 + if worker is not current and worker.isAlive():
51.1468 + try:
51.1469 + if timeout is None or timeout < 0:
51.1470 + worker.join()
51.1471 + else:
51.1472 + remaining_time = endtime - time.time()
51.1473 + if remaining_time > 0:
51.1474 + worker.join(remaining_time)
51.1475 + if worker.isAlive():
51.1476 + # We exhausted the timeout.
51.1477 + # Forcibly shut down the socket.
51.1478 + c = worker.conn
51.1479 + if c and not c.rfile.closed:
51.1480 + try:
51.1481 + c.socket.shutdown(socket.SHUT_RD)
51.1482 + except TypeError:
51.1483 + # pyOpenSSL sockets don't take an arg
51.1484 + c.socket.shutdown()
51.1485 + worker.join()
51.1486 + except (AssertionError,
51.1487 + # Ignore repeated Ctrl-C.
51.1488 + # See http://www.cherrypy.org/ticket/691.
51.1489 + KeyboardInterrupt), exc1:
51.1490 + pass
51.1491 +
51.1492 + def _get_qsize(self):
51.1493 + return self._queue.qsize()
51.1494 + qsize = property(_get_qsize)
51.1495 +
51.1496 +
51.1497 +
51.1498 +try:
51.1499 + import fcntl
51.1500 +except ImportError:
51.1501 + try:
51.1502 + from ctypes import windll, WinError
51.1503 + except ImportError:
51.1504 + def prevent_socket_inheritance(sock):
51.1505 + """Dummy function, since neither fcntl nor ctypes are available."""
51.1506 + pass
51.1507 + else:
51.1508 + def prevent_socket_inheritance(sock):
51.1509 + """Mark the given socket fd as non-inheritable (Windows)."""
51.1510 + if not windll.kernel32.SetHandleInformation(sock.fileno(), 1, 0):
51.1511 + raise WinError()
51.1512 +else:
51.1513 + def prevent_socket_inheritance(sock):
51.1514 + """Mark the given socket fd as non-inheritable (POSIX)."""
51.1515 + fd = sock.fileno()
51.1516 + old_flags = fcntl.fcntl(fd, fcntl.F_GETFD)
51.1517 + fcntl.fcntl(fd, fcntl.F_SETFD, old_flags | fcntl.FD_CLOEXEC)
51.1518 +
51.1519 +
51.1520 +class SSLAdapter(object):
51.1521 + """Base class for SSL driver library adapters.
51.1522 +
51.1523 + Required methods:
51.1524 +
51.1525 + * ``wrap(sock) -> (wrapped socket, ssl environ dict)``
51.1526 + * ``makefile(sock, mode='r', bufsize=DEFAULT_BUFFER_SIZE) -> socket file object``
51.1527 + """
51.1528 +
51.1529 + def __init__(self, certificate, private_key, certificate_chain=None):
51.1530 + self.certificate = certificate
51.1531 + self.private_key = private_key
51.1532 + self.certificate_chain = certificate_chain
51.1533 +
51.1534 + def wrap(self, sock):
51.1535 + raise NotImplemented
51.1536 +
51.1537 + def makefile(self, sock, mode='r', bufsize=DEFAULT_BUFFER_SIZE):
51.1538 + raise NotImplemented
51.1539 +
51.1540 +
51.1541 +class HTTPServer(object):
51.1542 + """An HTTP server."""
51.1543 +
51.1544 + _bind_addr = "127.0.0.1"
51.1545 + _interrupt = None
51.1546 +
51.1547 + gateway = None
51.1548 + """A Gateway instance."""
51.1549 +
51.1550 + minthreads = None
51.1551 + """The minimum number of worker threads to create (default 10)."""
51.1552 +
51.1553 + maxthreads = None
51.1554 + """The maximum number of worker threads to create (default -1 = no limit)."""
51.1555 +
51.1556 + server_name = None
51.1557 + """The name of the server; defaults to socket.gethostname()."""
51.1558 +
51.1559 + protocol = "HTTP/1.1"
51.1560 + """The version string to write in the Status-Line of all HTTP responses.
51.1561 +
51.1562 + For example, "HTTP/1.1" is the default. This also limits the supported
51.1563 + features used in the response."""
51.1564 +
51.1565 + request_queue_size = 5
51.1566 + """The 'backlog' arg to socket.listen(); max queued connections (default 5)."""
51.1567 +
51.1568 + shutdown_timeout = 5
51.1569 + """The total time, in seconds, to wait for worker threads to cleanly exit."""
51.1570 +
51.1571 + timeout = 10
51.1572 + """The timeout in seconds for accepted connections (default 10)."""
51.1573 +
51.1574 + version = "CherryPy/3.2.0"
51.1575 + """A version string for the HTTPServer."""
51.1576 +
51.1577 + software = None
51.1578 + """The value to set for the SERVER_SOFTWARE entry in the WSGI environ.
51.1579 +
51.1580 + If None, this defaults to ``'%s Server' % self.version``."""
51.1581 +
51.1582 + ready = False
51.1583 + """An internal flag which marks whether the socket is accepting connections."""
51.1584 +
51.1585 + max_request_header_size = 0
51.1586 + """The maximum size, in bytes, for request headers, or 0 for no limit."""
51.1587 +
51.1588 + max_request_body_size = 0
51.1589 + """The maximum size, in bytes, for request bodies, or 0 for no limit."""
51.1590 +
51.1591 + nodelay = True
51.1592 + """If True (the default since 3.1), sets the TCP_NODELAY socket option."""
51.1593 +
51.1594 + ConnectionClass = HTTPConnection
51.1595 + """The class to use for handling HTTP connections."""
51.1596 +
51.1597 + ssl_adapter = None
51.1598 + """An instance of SSLAdapter (or a subclass).
51.1599 +
51.1600 + You must have the corresponding SSL driver library installed."""
51.1601 +
51.1602 + def __init__(self, bind_addr, gateway, minthreads=10, maxthreads=-1,
51.1603 + server_name=None):
51.1604 + self.bind_addr = bind_addr
51.1605 + self.gateway = gateway
51.1606 +
51.1607 + self.requests = ThreadPool(self, min=minthreads or 1, max=maxthreads)
51.1608 +
51.1609 + if not server_name:
51.1610 + server_name = socket.gethostname()
51.1611 + self.server_name = server_name
51.1612 + self.clear_stats()
51.1613 +
51.1614 + def clear_stats(self):
51.1615 + self._start_time = None
51.1616 + self._run_time = 0
51.1617 + self.stats = {
51.1618 + 'Enabled': False,
51.1619 + 'Bind Address': lambda s: repr(self.bind_addr),
51.1620 + 'Run time': lambda s: (not s['Enabled']) and 0 or self.runtime(),
51.1621 + 'Accepts': 0,
51.1622 + 'Accepts/sec': lambda s: s['Accepts'] / self.runtime(),
51.1623 + 'Queue': lambda s: getattr(self.requests, "qsize", None),
51.1624 + 'Threads': lambda s: len(getattr(self.requests, "_threads", [])),
51.1625 + 'Threads Idle': lambda s: getattr(self.requests, "idle", None),
51.1626 + 'Socket Errors': 0,
51.1627 + 'Requests': lambda s: (not s['Enabled']) and 0 or sum([w['Requests'](w) for w
51.1628 + in s['Worker Threads'].values()], 0),
51.1629 + 'Bytes Read': lambda s: (not s['Enabled']) and 0 or sum([w['Bytes Read'](w) for w
51.1630 + in s['Worker Threads'].values()], 0),
51.1631 + 'Bytes Written': lambda s: (not s['Enabled']) and 0 or sum([w['Bytes Written'](w) for w
51.1632 + in s['Worker Threads'].values()], 0),
51.1633 + 'Work Time': lambda s: (not s['Enabled']) and 0 or sum([w['Work Time'](w) for w
51.1634 + in s['Worker Threads'].values()], 0),
51.1635 + 'Read Throughput': lambda s: (not s['Enabled']) and 0 or sum(
51.1636 + [w['Bytes Read'](w) / (w['Work Time'](w) or 1e-6)
51.1637 + for w in s['Worker Threads'].values()], 0),
51.1638 + 'Write Throughput': lambda s: (not s['Enabled']) and 0 or sum(
51.1639 + [w['Bytes Written'](w) / (w['Work Time'](w) or 1e-6)
51.1640 + for w in s['Worker Threads'].values()], 0),
51.1641 + 'Worker Threads': {},
51.1642 + }
51.1643 + logging.statistics["CherryPy HTTPServer %d" % id(self)] = self.stats
51.1644 +
51.1645 + def runtime(self):
51.1646 + if self._start_time is None:
51.1647 + return self._run_time
51.1648 + else:
51.1649 + return self._run_time + (time.time() - self._start_time)
51.1650 +
51.1651 + def __str__(self):
51.1652 + return "%s.%s(%r)" % (self.__module__, self.__class__.__name__,
51.1653 + self.bind_addr)
51.1654 +
51.1655 + def _get_bind_addr(self):
51.1656 + return self._bind_addr
51.1657 + def _set_bind_addr(self, value):
51.1658 + if isinstance(value, tuple) and value[0] in ('', None):
51.1659 + # Despite the socket module docs, using '' does not
51.1660 + # allow AI_PASSIVE to work. Passing None instead
51.1661 + # returns '0.0.0.0' like we want. In other words:
51.1662 + # host AI_PASSIVE result
51.1663 + # '' Y 192.168.x.y
51.1664 + # '' N 192.168.x.y
51.1665 + # None Y 0.0.0.0
51.1666 + # None N 127.0.0.1
51.1667 + # But since you can get the same effect with an explicit
51.1668 + # '0.0.0.0', we deny both the empty string and None as values.
51.1669 + raise ValueError("Host values of '' or None are not allowed. "
51.1670 + "Use '0.0.0.0' (IPv4) or '::' (IPv6) instead "
51.1671 + "to listen on all active interfaces.")
51.1672 + self._bind_addr = value
51.1673 + bind_addr = property(_get_bind_addr, _set_bind_addr,
51.1674 + doc="""The interface on which to listen for connections.
51.1675 +
51.1676 + For TCP sockets, a (host, port) tuple. Host values may be any IPv4
51.1677 + or IPv6 address, or any valid hostname. The string 'localhost' is a
51.1678 + synonym for '127.0.0.1' (or '::1', if your hosts file prefers IPv6).
51.1679 + The string '0.0.0.0' is a special IPv4 entry meaning "any active
51.1680 + interface" (INADDR_ANY), and '::' is the similar IN6ADDR_ANY for
51.1681 + IPv6. The empty string or None are not allowed.
51.1682 +
51.1683 + For UNIX sockets, supply the filename as a string.""")
51.1684 +
51.1685 + def start(self):
51.1686 + """Run the server forever."""
51.1687 + # We don't have to trap KeyboardInterrupt or SystemExit here,
51.1688 + # because cherrpy.server already does so, calling self.stop() for us.
51.1689 + # If you're using this server with another framework, you should
51.1690 + # trap those exceptions in whatever code block calls start().
51.1691 + self._interrupt = None
51.1692 +
51.1693 + if self.software is None:
51.1694 + self.software = "%s Server" % self.version
51.1695 +
51.1696 + # SSL backward compatibility
51.1697 + if (self.ssl_adapter is None and
51.1698 + getattr(self, 'ssl_certificate', None) and
51.1699 + getattr(self, 'ssl_private_key', None)):
51.1700 + warnings.warn(
51.1701 + "SSL attributes are deprecated in CherryPy 3.2, and will "
51.1702 + "be removed in CherryPy 3.3. Use an ssl_adapter attribute "
51.1703 + "instead.",
51.1704 + DeprecationWarning
51.1705 + )
51.1706 + try:
51.1707 + from cherrypy.wsgiserver.ssl_pyopenssl import pyOpenSSLAdapter
51.1708 + except ImportError:
51.1709 + pass
51.1710 + else:
51.1711 + self.ssl_adapter = pyOpenSSLAdapter(
51.1712 + self.ssl_certificate, self.ssl_private_key,
51.1713 + getattr(self, 'ssl_certificate_chain', None))
51.1714 +
51.1715 + # Select the appropriate socket
51.1716 + if isinstance(self.bind_addr, basestring):
51.1717 + # AF_UNIX socket
51.1718 +
51.1719 + # So we can reuse the socket...
51.1720 + try: os.unlink(self.bind_addr)
51.1721 + except: pass
51.1722 +
51.1723 + # So everyone can access the socket...
51.1724 + try: os.chmod(self.bind_addr, 0777)
51.1725 + except: pass
51.1726 +
51.1727 + info = [(socket.AF_UNIX, socket.SOCK_STREAM, 0, "", self.bind_addr)]
51.1728 + else:
51.1729 + # AF_INET or AF_INET6 socket
51.1730 + # Get the correct address family for our host (allows IPv6 addresses)
51.1731 + host, port = self.bind_addr
51.1732 + try:
51.1733 + info = socket.getaddrinfo(host, port, socket.AF_UNSPEC,
51.1734 + socket.SOCK_STREAM, 0, socket.AI_PASSIVE)
51.1735 + except socket.gaierror:
51.1736 + if ':' in self.bind_addr[0]:
51.1737 + info = [(socket.AF_INET6, socket.SOCK_STREAM,
51.1738 + 0, "", self.bind_addr + (0, 0))]
51.1739 + else:
51.1740 + info = [(socket.AF_INET, socket.SOCK_STREAM,
51.1741 + 0, "", self.bind_addr)]
51.1742 +
51.1743 + self.socket = None
51.1744 + msg = "No socket could be created"
51.1745 + for res in info:
51.1746 + af, socktype, proto, canonname, sa = res
51.1747 + try:
51.1748 + self.bind(af, socktype, proto)
51.1749 + except socket.error:
51.1750 + if self.socket:
51.1751 + self.socket.close()
51.1752 + self.socket = None
51.1753 + continue
51.1754 + break
51.1755 + if not self.socket:
51.1756 + raise socket.error(msg)
51.1757 +
51.1758 + # Timeout so KeyboardInterrupt can be caught on Win32
51.1759 + self.socket.settimeout(1)
51.1760 + self.socket.listen(self.request_queue_size)
51.1761 +
51.1762 + # Create worker threads
51.1763 + self.requests.start()
51.1764 +
51.1765 + self.ready = True
51.1766 + self._start_time = time.time()
51.1767 + while self.ready:
51.1768 + self.tick()
51.1769 + if self.interrupt:
51.1770 + while self.interrupt is True:
51.1771 + # Wait for self.stop() to complete. See _set_interrupt.
51.1772 + time.sleep(0.1)
51.1773 + if self.interrupt:
51.1774 + raise self.interrupt
51.1775 +
51.1776 + def bind(self, family, type, proto=0):
51.1777 + """Create (or recreate) the actual socket object."""
51.1778 + self.socket = socket.socket(family, type, proto)
51.1779 + prevent_socket_inheritance(self.socket)
51.1780 + self.socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
51.1781 + if self.nodelay and not isinstance(self.bind_addr, str):
51.1782 + self.socket.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)
51.1783 +
51.1784 + if self.ssl_adapter is not None:
51.1785 + self.socket = self.ssl_adapter.bind(self.socket)
51.1786 +
51.1787 + # If listening on the IPV6 any address ('::' = IN6ADDR_ANY),
51.1788 + # activate dual-stack. See http://www.cherrypy.org/ticket/871.
51.1789 + if (hasattr(socket, 'AF_INET6') and family == socket.AF_INET6
51.1790 + and self.bind_addr[0] in ('::', '::0', '::0.0.0.0')):
51.1791 + try:
51.1792 + self.socket.setsockopt(socket.IPPROTO_IPV6, socket.IPV6_V6ONLY, 0)
51.1793 + except (AttributeError, socket.error):
51.1794 + # Apparently, the socket option is not available in
51.1795 + # this machine's TCP stack
51.1796 + pass
51.1797 +
51.1798 + self.socket.bind(self.bind_addr)
51.1799 +
51.1800 + def tick(self):
51.1801 + """Accept a new connection and put it on the Queue."""
51.1802 + try:
51.1803 + s, addr = self.socket.accept()
51.1804 + if self.stats['Enabled']:
51.1805 + self.stats['Accepts'] += 1
51.1806 + if not self.ready:
51.1807 + return
51.1808 +
51.1809 + prevent_socket_inheritance(s)
51.1810 + if hasattr(s, 'settimeout'):
51.1811 + s.settimeout(self.timeout)
51.1812 +
51.1813 + makefile = CP_fileobject
51.1814 + ssl_env = {}
51.1815 + # if ssl cert and key are set, we try to be a secure HTTP server
51.1816 + if self.ssl_adapter is not None:
51.1817 + try:
51.1818 + s, ssl_env = self.ssl_adapter.wrap(s)
51.1819 + except NoSSLError:
51.1820 + msg = ("The client sent a plain HTTP request, but "
51.1821 + "this server only speaks HTTPS on this port.")
51.1822 + buf = ["%s 400 Bad Request\r\n" % self.protocol,
51.1823 + "Content-Length: %s\r\n" % len(msg),
51.1824 + "Content-Type: text/plain\r\n\r\n",
51.1825 + msg]
51.1826 +
51.1827 + wfile = CP_fileobject(s, "wb", DEFAULT_BUFFER_SIZE)
51.1828 + try:
51.1829 + wfile.sendall("".join(buf))
51.1830 + except socket.error, x:
51.1831 + if x.args[0] not in socket_errors_to_ignore:
51.1832 + raise
51.1833 + return
51.1834 + if not s:
51.1835 + return
51.1836 + makefile = self.ssl_adapter.makefile
51.1837 + # Re-apply our timeout since we may have a new socket object
51.1838 + if hasattr(s, 'settimeout'):
51.1839 + s.settimeout(self.timeout)
51.1840 +
51.1841 + conn = self.ConnectionClass(self, s, makefile)
51.1842 +
51.1843 + if not isinstance(self.bind_addr, basestring):
51.1844 + # optional values
51.1845 + # Until we do DNS lookups, omit REMOTE_HOST
51.1846 + if addr is None: # sometimes this can happen
51.1847 + # figure out if AF_INET or AF_INET6.
51.1848 + if len(s.getsockname()) == 2:
51.1849 + # AF_INET
51.1850 + addr = ('0.0.0.0', 0)
51.1851 + else:
51.1852 + # AF_INET6
51.1853 + addr = ('::', 0)
51.1854 + conn.remote_addr = addr[0]
51.1855 + conn.remote_port = addr[1]
51.1856 +
51.1857 + conn.ssl_env = ssl_env
51.1858 +
51.1859 + self.requests.put(conn)
51.1860 + except socket.timeout:
51.1861 + # The only reason for the timeout in start() is so we can
51.1862 + # notice keyboard interrupts on Win32, which don't interrupt
51.1863 + # accept() by default
51.1864 + return
51.1865 + except socket.error, x:
51.1866 + if self.stats['Enabled']:
51.1867 + self.stats['Socket Errors'] += 1
51.1868 + if x.args[0] in socket_error_eintr:
51.1869 + # I *think* this is right. EINTR should occur when a signal
51.1870 + # is received during the accept() call; all docs say retry
51.1871 + # the call, and I *think* I'm reading it right that Python
51.1872 + # will then go ahead and poll for and handle the signal
51.1873 + # elsewhere. See http://www.cherrypy.org/ticket/707.
51.1874 + return
51.1875 + if x.args[0] in socket_errors_nonblocking:
51.1876 + # Just try again. See http://www.cherrypy.org/ticket/479.
51.1877 + return
51.1878 + if x.args[0] in socket_errors_to_ignore:
51.1879 + # Our socket was closed.
51.1880 + # See http://www.cherrypy.org/ticket/686.
51.1881 + return
51.1882 + raise
51.1883 +
51.1884 + def _get_interrupt(self):
51.1885 + return self._interrupt
51.1886 + def _set_interrupt(self, interrupt):
51.1887 + self._interrupt = True
51.1888 + self.stop()
51.1889 + self._interrupt = interrupt
51.1890 + interrupt = property(_get_interrupt, _set_interrupt,
51.1891 + doc="Set this to an Exception instance to "
51.1892 + "interrupt the server.")
51.1893 +
51.1894 + def stop(self):
51.1895 + """Gracefully shutdown a server that is serving forever."""
51.1896 + self.ready = False
51.1897 + if self._start_time is not None:
51.1898 + self._run_time += (time.time() - self._start_time)
51.1899 + self._start_time = None
51.1900 +
51.1901 + sock = getattr(self, "socket", None)
51.1902 + if sock:
51.1903 + if not isinstance(self.bind_addr, basestring):
51.1904 + # Touch our own socket to make accept() return immediately.
51.1905 + try:
51.1906 + host, port = sock.getsockname()[:2]
51.1907 + except socket.error, x:
51.1908 + if x.args[0] not in socket_errors_to_ignore:
51.1909 + # Changed to use error code and not message
51.1910 + # See http://www.cherrypy.org/ticket/860.
51.1911 + raise
51.1912 + else:
51.1913 + # Note that we're explicitly NOT using AI_PASSIVE,
51.1914 + # here, because we want an actual IP to touch.
51.1915 + # localhost won't work if we've bound to a public IP,
51.1916 + # but it will if we bound to '0.0.0.0' (INADDR_ANY).
51.1917 + for res in socket.getaddrinfo(host, port, socket.AF_UNSPEC,
51.1918 + socket.SOCK_STREAM):
51.1919 + af, socktype, proto, canonname, sa = res
51.1920 + s = None
51.1921 + try:
51.1922 + s = socket.socket(af, socktype, proto)
51.1923 + # See http://groups.google.com/group/cherrypy-users/
51.1924 + # browse_frm/thread/bbfe5eb39c904fe0
51.1925 + s.settimeout(1.0)
51.1926 + s.connect((host, port))
51.1927 + s.close()
51.1928 + except socket.error:
51.1929 + if s:
51.1930 + s.close()
51.1931 + if hasattr(sock, "close"):
51.1932 + sock.close()
51.1933 + self.socket = None
51.1934 +
51.1935 + self.requests.stop(self.shutdown_timeout)
51.1936 +
51.1937 +
51.1938 +class Gateway(object):
51.1939 +
51.1940 + def __init__(self, req):
51.1941 + self.req = req
51.1942 +
51.1943 + def respond(self):
51.1944 + raise NotImplemented
51.1945 +
51.1946 +
51.1947 +# These may either be wsgiserver.SSLAdapter subclasses or the string names
51.1948 +# of such classes (in which case they will be lazily loaded).
51.1949 +ssl_adapters = {
51.1950 + 'builtin': 'cherrypy.wsgiserver.ssl_builtin.BuiltinSSLAdapter',
51.1951 + 'pyopenssl': 'cherrypy.wsgiserver.ssl_pyopenssl.pyOpenSSLAdapter',
51.1952 + }
51.1953 +
51.1954 +def get_ssl_adapter_class(name='pyopenssl'):
51.1955 + adapter = ssl_adapters[name.lower()]
51.1956 + if isinstance(adapter, basestring):
51.1957 + last_dot = adapter.rfind(".")
51.1958 + attr_name = adapter[last_dot + 1:]
51.1959 + mod_path = adapter[:last_dot]
51.1960 +
51.1961 + try:
51.1962 + mod = sys.modules[mod_path]
51.1963 + if mod is None:
51.1964 + raise KeyError()
51.1965 + except KeyError:
51.1966 + # The last [''] is important.
51.1967 + mod = __import__(mod_path, globals(), locals(), [''])
51.1968 +
51.1969 + # Let an AttributeError propagate outward.
51.1970 + try:
51.1971 + adapter = getattr(mod, attr_name)
51.1972 + except AttributeError:
51.1973 + raise AttributeError("'%s' object has no attribute '%s'"
51.1974 + % (mod_path, attr_name))
51.1975 +
51.1976 + return adapter
51.1977 +
51.1978 +# -------------------------------- WSGI Stuff -------------------------------- #
51.1979 +
51.1980 +
51.1981 +class CherryPyWSGIServer(HTTPServer):
51.1982 +
51.1983 + wsgi_version = (1, 0)
51.1984 +
51.1985 + def __init__(self, bind_addr, wsgi_app, numthreads=10, server_name=None,
51.1986 + max=-1, request_queue_size=5, timeout=10, shutdown_timeout=5):
51.1987 + self.requests = ThreadPool(self, min=numthreads or 1, max=max)
51.1988 + self.wsgi_app = wsgi_app
51.1989 + self.gateway = wsgi_gateways[self.wsgi_version]
51.1990 +
51.1991 + self.bind_addr = bind_addr
51.1992 + if not server_name:
51.1993 + server_name = socket.gethostname()
51.1994 + self.server_name = server_name
51.1995 + self.request_queue_size = request_queue_size
51.1996 +
51.1997 + self.timeout = timeout
51.1998 + self.shutdown_timeout = shutdown_timeout
51.1999 + self.clear_stats()
51.2000 +
51.2001 + def _get_numthreads(self):
51.2002 + return self.requests.min
51.2003 + def _set_numthreads(self, value):
51.2004 + self.requests.min = value
51.2005 + numthreads = property(_get_numthreads, _set_numthreads)
51.2006 +
51.2007 +
51.2008 +class WSGIGateway(Gateway):
51.2009 +
51.2010 + def __init__(self, req):
51.2011 + self.req = req
51.2012 + self.started_response = False
51.2013 + self.env = self.get_environ()
51.2014 + self.remaining_bytes_out = None
51.2015 +
51.2016 + def get_environ(self):
51.2017 + """Return a new environ dict targeting the given wsgi.version"""
51.2018 + raise NotImplemented
51.2019 +
51.2020 + def respond(self):
51.2021 + response = self.req.server.wsgi_app(self.env, self.start_response)
51.2022 + try:
51.2023 + for chunk in response:
51.2024 + # "The start_response callable must not actually transmit
51.2025 + # the response headers. Instead, it must store them for the
51.2026 + # server or gateway to transmit only after the first
51.2027 + # iteration of the application return value that yields
51.2028 + # a NON-EMPTY string, or upon the application's first
51.2029 + # invocation of the write() callable." (PEP 333)
51.2030 + if chunk:
51.2031 + if isinstance(chunk, unicode):
51.2032 + chunk = chunk.encode('ISO-8859-1')
51.2033 + self.write(chunk)
51.2034 + finally:
51.2035 + if hasattr(response, "close"):
51.2036 + response.close()
51.2037 +
51.2038 + def start_response(self, status, headers, exc_info = None):
51.2039 + """WSGI callable to begin the HTTP response."""
51.2040 + # "The application may call start_response more than once,
51.2041 + # if and only if the exc_info argument is provided."
51.2042 + if self.started_response and not exc_info:
51.2043 + raise AssertionError("WSGI start_response called a second "
51.2044 + "time with no exc_info.")
51.2045 + self.started_response = True
51.2046 +
51.2047 + # "if exc_info is provided, and the HTTP headers have already been
51.2048 + # sent, start_response must raise an error, and should raise the
51.2049 + # exc_info tuple."
51.2050 + if self.req.sent_headers:
51.2051 + try:
51.2052 + raise exc_info[0], exc_info[1], exc_info[2]
51.2053 + finally:
51.2054 + exc_info = None
51.2055 +
51.2056 + self.req.status = status
51.2057 + for k, v in headers:
51.2058 + if not isinstance(k, str):
51.2059 + raise TypeError("WSGI response header key %r is not a byte string." % k)
51.2060 + if not isinstance(v, str):
51.2061 + raise TypeError("WSGI response header value %r is not a byte string." % v)
51.2062 + if k.lower() == 'content-length':
51.2063 + self.remaining_bytes_out = int(v)
51.2064 + self.req.outheaders.extend(headers)
51.2065 +
51.2066 + return self.write
51.2067 +
51.2068 + def write(self, chunk):
51.2069 + """WSGI callable to write unbuffered data to the client.
51.2070 +
51.2071 + This method is also used internally by start_response (to write
51.2072 + data from the iterable returned by the WSGI application).
51.2073 + """
51.2074 + if not self.started_response:
51.2075 + raise AssertionError("WSGI write called before start_response.")
51.2076 +
51.2077 + chunklen = len(chunk)
51.2078 + rbo = self.remaining_bytes_out
51.2079 + if rbo is not None and chunklen > rbo:
51.2080 + if not self.req.sent_headers:
51.2081 + # Whew. We can send a 500 to the client.
51.2082 + self.req.simple_response("500 Internal Server Error",
51.2083 + "The requested resource returned more bytes than the "
51.2084 + "declared Content-Length.")
51.2085 + else:
51.2086 + # Dang. We have probably already sent data. Truncate the chunk
51.2087 + # to fit (so the client doesn't hang) and raise an error later.
51.2088 + chunk = chunk[:rbo]
51.2089 +
51.2090 + if not self.req.sent_headers:
51.2091 + self.req.sent_headers = True
51.2092 + self.req.send_headers()
51.2093 +
51.2094 + self.req.write(chunk)
51.2095 +
51.2096 + if rbo is not None:
51.2097 + rbo -= chunklen
51.2098 + if rbo < 0:
51.2099 + raise ValueError(
51.2100 + "Response body exceeds the declared Content-Length.")
51.2101 +
51.2102 +
51.2103 +class WSGIGateway_10(WSGIGateway):
51.2104 +
51.2105 + def get_environ(self):
51.2106 + """Return a new environ dict targeting the given wsgi.version"""
51.2107 + req = self.req
51.2108 + env = {
51.2109 + # set a non-standard environ entry so the WSGI app can know what
51.2110 + # the *real* server protocol is (and what features to support).
51.2111 + # See http://www.faqs.org/rfcs/rfc2145.html.
51.2112 + 'ACTUAL_SERVER_PROTOCOL': req.server.protocol,
51.2113 + 'PATH_INFO': req.path,
51.2114 + 'QUERY_STRING': req.qs,
51.2115 + 'REMOTE_ADDR': req.conn.remote_addr or '',
51.2116 + 'REMOTE_PORT': str(req.conn.remote_port or ''),
51.2117 + 'REQUEST_METHOD': req.method,
51.2118 + 'REQUEST_URI': req.uri,
51.2119 + 'SCRIPT_NAME': '',
51.2120 + 'SERVER_NAME': req.server.server_name,
51.2121 + # Bah. "SERVER_PROTOCOL" is actually the REQUEST protocol.
51.2122 + 'SERVER_PROTOCOL': req.request_protocol,
51.2123 + 'SERVER_SOFTWARE': req.server.software,
51.2124 + 'wsgi.errors': sys.stderr,
51.2125 + 'wsgi.input': req.rfile,
51.2126 + 'wsgi.multiprocess': False,
51.2127 + 'wsgi.multithread': True,
51.2128 + 'wsgi.run_once': False,
51.2129 + 'wsgi.url_scheme': req.scheme,
51.2130 + 'wsgi.version': (1, 0),
51.2131 + }
51.2132 +
51.2133 + if isinstance(req.server.bind_addr, basestring):
51.2134 + # AF_UNIX. This isn't really allowed by WSGI, which doesn't
51.2135 + # address unix domain sockets. But it's better than nothing.
51.2136 + env["SERVER_PORT"] = ""
51.2137 + else:
51.2138 + env["SERVER_PORT"] = str(req.server.bind_addr[1])
51.2139 +
51.2140 + # Request headers
51.2141 + for k, v in req.inheaders.iteritems():
51.2142 + env["HTTP_" + k.upper().replace("-", "_")] = v
51.2143 +
51.2144 + # CONTENT_TYPE/CONTENT_LENGTH
51.2145 + ct = env.pop("HTTP_CONTENT_TYPE", None)
51.2146 + if ct is not None:
51.2147 + env["CONTENT_TYPE"] = ct
51.2148 + cl = env.pop("HTTP_CONTENT_LENGTH", None)
51.2149 + if cl is not None:
51.2150 + env["CONTENT_LENGTH"] = cl
51.2151 +
51.2152 + if req.conn.ssl_env:
51.2153 + env.update(req.conn.ssl_env)
51.2154 +
51.2155 + return env
51.2156 +
51.2157 +
51.2158 +class WSGIGateway_u0(WSGIGateway_10):
51.2159 +
51.2160 + def get_environ(self):
51.2161 + """Return a new environ dict targeting the given wsgi.version"""
51.2162 + req = self.req
51.2163 + env_10 = WSGIGateway_10.get_environ(self)
51.2164 + env = dict([(k.decode('ISO-8859-1'), v) for k, v in env_10.iteritems()])
51.2165 + env[u'wsgi.version'] = ('u', 0)
51.2166 +
51.2167 + # Request-URI
51.2168 + env.setdefault(u'wsgi.url_encoding', u'utf-8')
51.2169 + try:
51.2170 + for key in [u"PATH_INFO", u"SCRIPT_NAME", u"QUERY_STRING"]:
51.2171 + env[key] = env_10[str(key)].decode(env[u'wsgi.url_encoding'])
51.2172 + except UnicodeDecodeError:
51.2173 + # Fall back to latin 1 so apps can transcode if needed.
51.2174 + env[u'wsgi.url_encoding'] = u'ISO-8859-1'
51.2175 + for key in [u"PATH_INFO", u"SCRIPT_NAME", u"QUERY_STRING"]:
51.2176 + env[key] = env_10[str(key)].decode(env[u'wsgi.url_encoding'])
51.2177 +
51.2178 + for k, v in sorted(env.items()):
51.2179 + if isinstance(v, str) and k not in ('REQUEST_URI', 'wsgi.input'):
51.2180 + env[k] = v.decode('ISO-8859-1')
51.2181 +
51.2182 + return env
51.2183 +
51.2184 +wsgi_gateways = {
51.2185 + (1, 0): WSGIGateway_10,
51.2186 + ('u', 0): WSGIGateway_u0,
51.2187 +}
51.2188 +
51.2189 +class WSGIPathInfoDispatcher(object):
51.2190 + """A WSGI dispatcher for dispatch based on the PATH_INFO.
51.2191 +
51.2192 + apps: a dict or list of (path_prefix, app) pairs.
51.2193 + """
51.2194 +
51.2195 + def __init__(self, apps):
51.2196 + try:
51.2197 + apps = apps.items()
51.2198 + except AttributeError:
51.2199 + pass
51.2200 +
51.2201 + # Sort the apps by len(path), descending
51.2202 + apps.sort(cmp=lambda x,y: cmp(len(x[0]), len(y[0])))
51.2203 + apps.reverse()
51.2204 +
51.2205 + # The path_prefix strings must start, but not end, with a slash.
51.2206 + # Use "" instead of "/".
51.2207 + self.apps = [(p.rstrip("/"), a) for p, a in apps]
51.2208 +
51.2209 + def __call__(self, environ, start_response):
51.2210 + path = environ["PATH_INFO"] or "/"
51.2211 + for p, app in self.apps:
51.2212 + # The apps list should be sorted by length, descending.
51.2213 + if path.startswith(p + "/") or path == p:
51.2214 + environ = environ.copy()
51.2215 + environ["SCRIPT_NAME"] = environ["SCRIPT_NAME"] + p
51.2216 + environ["PATH_INFO"] = path[len(p):]
51.2217 + return app(environ, start_response)
51.2218 +
51.2219 + start_response('404 Not Found', [('Content-Type', 'text/plain'),
51.2220 + ('Content-Length', '0')])
51.2221 + return ['']
51.2222 +
52.1 --- /dev/null Thu Jan 01 00:00:00 1970 +0000
52.2 +++ b/OpenSecurity/install/web.py-0.37/build/lib/web/wsgiserver/ssl_builtin.py Mon Dec 02 14:02:05 2013 +0100
52.3 @@ -0,0 +1,72 @@
52.4 +"""A library for integrating Python's builtin ``ssl`` library with CherryPy.
52.5 +
52.6 +The ssl module must be importable for SSL functionality.
52.7 +
52.8 +To use this module, set ``CherryPyWSGIServer.ssl_adapter`` to an instance of
52.9 +``BuiltinSSLAdapter``.
52.10 +"""
52.11 +
52.12 +try:
52.13 + import ssl
52.14 +except ImportError:
52.15 + ssl = None
52.16 +
52.17 +from cherrypy import wsgiserver
52.18 +
52.19 +
52.20 +class BuiltinSSLAdapter(wsgiserver.SSLAdapter):
52.21 + """A wrapper for integrating Python's builtin ssl module with CherryPy."""
52.22 +
52.23 + certificate = None
52.24 + """The filename of the server SSL certificate."""
52.25 +
52.26 + private_key = None
52.27 + """The filename of the server's private key file."""
52.28 +
52.29 + def __init__(self, certificate, private_key, certificate_chain=None):
52.30 + if ssl is None:
52.31 + raise ImportError("You must install the ssl module to use HTTPS.")
52.32 + self.certificate = certificate
52.33 + self.private_key = private_key
52.34 + self.certificate_chain = certificate_chain
52.35 +
52.36 + def bind(self, sock):
52.37 + """Wrap and return the given socket."""
52.38 + return sock
52.39 +
52.40 + def wrap(self, sock):
52.41 + """Wrap and return the given socket, plus WSGI environ entries."""
52.42 + try:
52.43 + s = ssl.wrap_socket(sock, do_handshake_on_connect=True,
52.44 + server_side=True, certfile=self.certificate,
52.45 + keyfile=self.private_key, ssl_version=ssl.PROTOCOL_SSLv23)
52.46 + except ssl.SSLError, e:
52.47 + if e.errno == ssl.SSL_ERROR_EOF:
52.48 + # This is almost certainly due to the cherrypy engine
52.49 + # 'pinging' the socket to assert it's connectable;
52.50 + # the 'ping' isn't SSL.
52.51 + return None, {}
52.52 + elif e.errno == ssl.SSL_ERROR_SSL:
52.53 + if e.args[1].endswith('http request'):
52.54 + # The client is speaking HTTP to an HTTPS server.
52.55 + raise wsgiserver.NoSSLError
52.56 + raise
52.57 + return s, self.get_environ(s)
52.58 +
52.59 + # TODO: fill this out more with mod ssl env
52.60 + def get_environ(self, sock):
52.61 + """Create WSGI environ entries to be merged into each request."""
52.62 + cipher = sock.cipher()
52.63 + ssl_environ = {
52.64 + "wsgi.url_scheme": "https",
52.65 + "HTTPS": "on",
52.66 + 'SSL_PROTOCOL': cipher[1],
52.67 + 'SSL_CIPHER': cipher[0]
52.68 +## SSL_VERSION_INTERFACE string The mod_ssl program version
52.69 +## SSL_VERSION_LIBRARY string The OpenSSL program version
52.70 + }
52.71 + return ssl_environ
52.72 +
52.73 + def makefile(self, sock, mode='r', bufsize=-1):
52.74 + return wsgiserver.CP_fileobject(sock, mode, bufsize)
52.75 +
53.1 --- /dev/null Thu Jan 01 00:00:00 1970 +0000
53.2 +++ b/OpenSecurity/install/web.py-0.37/build/lib/web/wsgiserver/ssl_pyopenssl.py Mon Dec 02 14:02:05 2013 +0100
53.3 @@ -0,0 +1,256 @@
53.4 +"""A library for integrating pyOpenSSL with CherryPy.
53.5 +
53.6 +The OpenSSL module must be importable for SSL functionality.
53.7 +You can obtain it from http://pyopenssl.sourceforge.net/
53.8 +
53.9 +To use this module, set CherryPyWSGIServer.ssl_adapter to an instance of
53.10 +SSLAdapter. There are two ways to use SSL:
53.11 +
53.12 +Method One
53.13 +----------
53.14 +
53.15 + * ``ssl_adapter.context``: an instance of SSL.Context.
53.16 +
53.17 +If this is not None, it is assumed to be an SSL.Context instance,
53.18 +and will be passed to SSL.Connection on bind(). The developer is
53.19 +responsible for forming a valid Context object. This approach is
53.20 +to be preferred for more flexibility, e.g. if the cert and key are
53.21 +streams instead of files, or need decryption, or SSL.SSLv3_METHOD
53.22 +is desired instead of the default SSL.SSLv23_METHOD, etc. Consult
53.23 +the pyOpenSSL documentation for complete options.
53.24 +
53.25 +Method Two (shortcut)
53.26 +---------------------
53.27 +
53.28 + * ``ssl_adapter.certificate``: the filename of the server SSL certificate.
53.29 + * ``ssl_adapter.private_key``: the filename of the server's private key file.
53.30 +
53.31 +Both are None by default. If ssl_adapter.context is None, but .private_key
53.32 +and .certificate are both given and valid, they will be read, and the
53.33 +context will be automatically created from them.
53.34 +"""
53.35 +
53.36 +import socket
53.37 +import threading
53.38 +import time
53.39 +
53.40 +from cherrypy import wsgiserver
53.41 +
53.42 +try:
53.43 + from OpenSSL import SSL
53.44 + from OpenSSL import crypto
53.45 +except ImportError:
53.46 + SSL = None
53.47 +
53.48 +
53.49 +class SSL_fileobject(wsgiserver.CP_fileobject):
53.50 + """SSL file object attached to a socket object."""
53.51 +
53.52 + ssl_timeout = 3
53.53 + ssl_retry = .01
53.54 +
53.55 + def _safe_call(self, is_reader, call, *args, **kwargs):
53.56 + """Wrap the given call with SSL error-trapping.
53.57 +
53.58 + is_reader: if False EOF errors will be raised. If True, EOF errors
53.59 + will return "" (to emulate normal sockets).
53.60 + """
53.61 + start = time.time()
53.62 + while True:
53.63 + try:
53.64 + return call(*args, **kwargs)
53.65 + except SSL.WantReadError:
53.66 + # Sleep and try again. This is dangerous, because it means
53.67 + # the rest of the stack has no way of differentiating
53.68 + # between a "new handshake" error and "client dropped".
53.69 + # Note this isn't an endless loop: there's a timeout below.
53.70 + time.sleep(self.ssl_retry)
53.71 + except SSL.WantWriteError:
53.72 + time.sleep(self.ssl_retry)
53.73 + except SSL.SysCallError, e:
53.74 + if is_reader and e.args == (-1, 'Unexpected EOF'):
53.75 + return ""
53.76 +
53.77 + errnum = e.args[0]
53.78 + if is_reader and errnum in wsgiserver.socket_errors_to_ignore:
53.79 + return ""
53.80 + raise socket.error(errnum)
53.81 + except SSL.Error, e:
53.82 + if is_reader and e.args == (-1, 'Unexpected EOF'):
53.83 + return ""
53.84 +
53.85 + thirdarg = None
53.86 + try:
53.87 + thirdarg = e.args[0][0][2]
53.88 + except IndexError:
53.89 + pass
53.90 +
53.91 + if thirdarg == 'http request':
53.92 + # The client is talking HTTP to an HTTPS server.
53.93 + raise wsgiserver.NoSSLError()
53.94 +
53.95 + raise wsgiserver.FatalSSLAlert(*e.args)
53.96 + except:
53.97 + raise
53.98 +
53.99 + if time.time() - start > self.ssl_timeout:
53.100 + raise socket.timeout("timed out")
53.101 +
53.102 + def recv(self, *args, **kwargs):
53.103 + buf = []
53.104 + r = super(SSL_fileobject, self).recv
53.105 + while True:
53.106 + data = self._safe_call(True, r, *args, **kwargs)
53.107 + buf.append(data)
53.108 + p = self._sock.pending()
53.109 + if not p:
53.110 + return "".join(buf)
53.111 +
53.112 + def sendall(self, *args, **kwargs):
53.113 + return self._safe_call(False, super(SSL_fileobject, self).sendall,
53.114 + *args, **kwargs)
53.115 +
53.116 + def send(self, *args, **kwargs):
53.117 + return self._safe_call(False, super(SSL_fileobject, self).send,
53.118 + *args, **kwargs)
53.119 +
53.120 +
53.121 +class SSLConnection:
53.122 + """A thread-safe wrapper for an SSL.Connection.
53.123 +
53.124 + ``*args``: the arguments to create the wrapped ``SSL.Connection(*args)``.
53.125 + """
53.126 +
53.127 + def __init__(self, *args):
53.128 + self._ssl_conn = SSL.Connection(*args)
53.129 + self._lock = threading.RLock()
53.130 +
53.131 + for f in ('get_context', 'pending', 'send', 'write', 'recv', 'read',
53.132 + 'renegotiate', 'bind', 'listen', 'connect', 'accept',
53.133 + 'setblocking', 'fileno', 'close', 'get_cipher_list',
53.134 + 'getpeername', 'getsockname', 'getsockopt', 'setsockopt',
53.135 + 'makefile', 'get_app_data', 'set_app_data', 'state_string',
53.136 + 'sock_shutdown', 'get_peer_certificate', 'want_read',
53.137 + 'want_write', 'set_connect_state', 'set_accept_state',
53.138 + 'connect_ex', 'sendall', 'settimeout', 'gettimeout'):
53.139 + exec("""def %s(self, *args):
53.140 + self._lock.acquire()
53.141 + try:
53.142 + return self._ssl_conn.%s(*args)
53.143 + finally:
53.144 + self._lock.release()
53.145 +""" % (f, f))
53.146 +
53.147 + def shutdown(self, *args):
53.148 + self._lock.acquire()
53.149 + try:
53.150 + # pyOpenSSL.socket.shutdown takes no args
53.151 + return self._ssl_conn.shutdown()
53.152 + finally:
53.153 + self._lock.release()
53.154 +
53.155 +
53.156 +class pyOpenSSLAdapter(wsgiserver.SSLAdapter):
53.157 + """A wrapper for integrating pyOpenSSL with CherryPy."""
53.158 +
53.159 + context = None
53.160 + """An instance of SSL.Context."""
53.161 +
53.162 + certificate = None
53.163 + """The filename of the server SSL certificate."""
53.164 +
53.165 + private_key = None
53.166 + """The filename of the server's private key file."""
53.167 +
53.168 + certificate_chain = None
53.169 + """Optional. The filename of CA's intermediate certificate bundle.
53.170 +
53.171 + This is needed for cheaper "chained root" SSL certificates, and should be
53.172 + left as None if not required."""
53.173 +
53.174 + def __init__(self, certificate, private_key, certificate_chain=None):
53.175 + if SSL is None:
53.176 + raise ImportError("You must install pyOpenSSL to use HTTPS.")
53.177 +
53.178 + self.context = None
53.179 + self.certificate = certificate
53.180 + self.private_key = private_key
53.181 + self.certificate_chain = certificate_chain
53.182 + self._environ = None
53.183 +
53.184 + def bind(self, sock):
53.185 + """Wrap and return the given socket."""
53.186 + if self.context is None:
53.187 + self.context = self.get_context()
53.188 + conn = SSLConnection(self.context, sock)
53.189 + self._environ = self.get_environ()
53.190 + return conn
53.191 +
53.192 + def wrap(self, sock):
53.193 + """Wrap and return the given socket, plus WSGI environ entries."""
53.194 + return sock, self._environ.copy()
53.195 +
53.196 + def get_context(self):
53.197 + """Return an SSL.Context from self attributes."""
53.198 + # See http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/442473
53.199 + c = SSL.Context(SSL.SSLv23_METHOD)
53.200 + c.use_privatekey_file(self.private_key)
53.201 + if self.certificate_chain:
53.202 + c.load_verify_locations(self.certificate_chain)
53.203 + c.use_certificate_file(self.certificate)
53.204 + return c
53.205 +
53.206 + def get_environ(self):
53.207 + """Return WSGI environ entries to be merged into each request."""
53.208 + ssl_environ = {
53.209 + "HTTPS": "on",
53.210 + # pyOpenSSL doesn't provide access to any of these AFAICT
53.211 +## 'SSL_PROTOCOL': 'SSLv2',
53.212 +## SSL_CIPHER string The cipher specification name
53.213 +## SSL_VERSION_INTERFACE string The mod_ssl program version
53.214 +## SSL_VERSION_LIBRARY string The OpenSSL program version
53.215 + }
53.216 +
53.217 + if self.certificate:
53.218 + # Server certificate attributes
53.219 + cert = open(self.certificate, 'rb').read()
53.220 + cert = crypto.load_certificate(crypto.FILETYPE_PEM, cert)
53.221 + ssl_environ.update({
53.222 + 'SSL_SERVER_M_VERSION': cert.get_version(),
53.223 + 'SSL_SERVER_M_SERIAL': cert.get_serial_number(),
53.224 +## 'SSL_SERVER_V_START': Validity of server's certificate (start time),
53.225 +## 'SSL_SERVER_V_END': Validity of server's certificate (end time),
53.226 + })
53.227 +
53.228 + for prefix, dn in [("I", cert.get_issuer()),
53.229 + ("S", cert.get_subject())]:
53.230 + # X509Name objects don't seem to have a way to get the
53.231 + # complete DN string. Use str() and slice it instead,
53.232 + # because str(dn) == "<X509Name object '/C=US/ST=...'>"
53.233 + dnstr = str(dn)[18:-2]
53.234 +
53.235 + wsgikey = 'SSL_SERVER_%s_DN' % prefix
53.236 + ssl_environ[wsgikey] = dnstr
53.237 +
53.238 + # The DN should be of the form: /k1=v1/k2=v2, but we must allow
53.239 + # for any value to contain slashes itself (in a URL).
53.240 + while dnstr:
53.241 + pos = dnstr.rfind("=")
53.242 + dnstr, value = dnstr[:pos], dnstr[pos + 1:]
53.243 + pos = dnstr.rfind("/")
53.244 + dnstr, key = dnstr[:pos], dnstr[pos + 1:]
53.245 + if key and value:
53.246 + wsgikey = 'SSL_SERVER_%s_DN_%s' % (prefix, key)
53.247 + ssl_environ[wsgikey] = value
53.248 +
53.249 + return ssl_environ
53.250 +
53.251 + def makefile(self, sock, mode='r', bufsize=-1):
53.252 + if SSL and isinstance(sock, SSL.ConnectionType):
53.253 + timeout = sock.gettimeout()
53.254 + f = SSL_fileobject(sock, mode, bufsize)
53.255 + f.ssl_timeout = timeout
53.256 + return f
53.257 + else:
53.258 + return wsgiserver.CP_fileobject(sock, mode, bufsize)
53.259 +
54.1 --- /dev/null Thu Jan 01 00:00:00 1970 +0000
54.2 +++ b/OpenSecurity/install/web.py-0.37/setup.py Mon Dec 02 14:02:05 2013 +0100
54.3 @@ -0,0 +1,20 @@
54.4 +#!/usr/bin/env python
54.5 +
54.6 +# ...
54.7 +
54.8 +from distutils.core import setup
54.9 +from web import __version__
54.10 +
54.11 +setup(name='web.py',
54.12 + version=__version__,
54.13 + description='web.py: makes web apps',
54.14 + author='Aaron Swartz',
54.15 + author_email='me@aaronsw.com',
54.16 + maintainer='Anand Chitipothu',
54.17 + maintainer_email='anandology@gmail.com',
54.18 + url=' http://webpy.org/',
54.19 + packages=['web', 'web.wsgiserver', 'web.contrib'],
54.20 + long_description="Think about the ideal way to write a web app. Write the code to make it happen.",
54.21 + license="Public domain",
54.22 + platforms=["any"],
54.23 + )
55.1 --- /dev/null Thu Jan 01 00:00:00 1970 +0000
55.2 +++ b/OpenSecurity/install/web.py-0.37/web/__init__.py Mon Dec 02 14:02:05 2013 +0100
55.3 @@ -0,0 +1,33 @@
55.4 +#!/usr/bin/env python
55.5 +"""web.py: makes web apps (http://webpy.org)"""
55.6 +
55.7 +from __future__ import generators
55.8 +
55.9 +__version__ = "0.37"
55.10 +__author__ = [
55.11 + "Aaron Swartz <me@aaronsw.com>",
55.12 + "Anand Chitipothu <anandology@gmail.com>"
55.13 +]
55.14 +__license__ = "public domain"
55.15 +__contributors__ = "see http://webpy.org/changes"
55.16 +
55.17 +import utils, db, net, wsgi, http, webapi, httpserver, debugerror
55.18 +import template, form
55.19 +
55.20 +import session
55.21 +
55.22 +from utils import *
55.23 +from db import *
55.24 +from net import *
55.25 +from wsgi import *
55.26 +from http import *
55.27 +from webapi import *
55.28 +from httpserver import *
55.29 +from debugerror import *
55.30 +from application import *
55.31 +from browser import *
55.32 +try:
55.33 + import webopenid as openid
55.34 +except ImportError:
55.35 + pass # requires openid module
55.36 +
56.1 Binary file OpenSecurity/install/web.py-0.37/web/__init__.pyc has changed
57.1 --- /dev/null Thu Jan 01 00:00:00 1970 +0000
57.2 +++ b/OpenSecurity/install/web.py-0.37/web/application.py Mon Dec 02 14:02:05 2013 +0100
57.3 @@ -0,0 +1,687 @@
57.4 +"""
57.5 +Web application
57.6 +(from web.py)
57.7 +"""
57.8 +import webapi as web
57.9 +import webapi, wsgi, utils
57.10 +import debugerror
57.11 +import httpserver
57.12 +
57.13 +from utils import lstrips, safeunicode
57.14 +import sys
57.15 +
57.16 +import urllib
57.17 +import traceback
57.18 +import itertools
57.19 +import os
57.20 +import types
57.21 +from exceptions import SystemExit
57.22 +
57.23 +try:
57.24 + import wsgiref.handlers
57.25 +except ImportError:
57.26 + pass # don't break people with old Pythons
57.27 +
57.28 +__all__ = [
57.29 + "application", "auto_application",
57.30 + "subdir_application", "subdomain_application",
57.31 + "loadhook", "unloadhook",
57.32 + "autodelegate"
57.33 +]
57.34 +
57.35 +class application:
57.36 + """
57.37 + Application to delegate requests based on path.
57.38 +
57.39 + >>> urls = ("/hello", "hello")
57.40 + >>> app = application(urls, globals())
57.41 + >>> class hello:
57.42 + ... def GET(self): return "hello"
57.43 + >>>
57.44 + >>> app.request("/hello").data
57.45 + 'hello'
57.46 + """
57.47 + def __init__(self, mapping=(), fvars={}, autoreload=None):
57.48 + if autoreload is None:
57.49 + autoreload = web.config.get('debug', False)
57.50 + self.init_mapping(mapping)
57.51 + self.fvars = fvars
57.52 + self.processors = []
57.53 +
57.54 + self.add_processor(loadhook(self._load))
57.55 + self.add_processor(unloadhook(self._unload))
57.56 +
57.57 + if autoreload:
57.58 + def main_module_name():
57.59 + mod = sys.modules['__main__']
57.60 + file = getattr(mod, '__file__', None) # make sure this works even from python interpreter
57.61 + return file and os.path.splitext(os.path.basename(file))[0]
57.62 +
57.63 + def modname(fvars):
57.64 + """find name of the module name from fvars."""
57.65 + file, name = fvars.get('__file__'), fvars.get('__name__')
57.66 + if file is None or name is None:
57.67 + return None
57.68 +
57.69 + if name == '__main__':
57.70 + # Since the __main__ module can't be reloaded, the module has
57.71 + # to be imported using its file name.
57.72 + name = main_module_name()
57.73 + return name
57.74 +
57.75 + mapping_name = utils.dictfind(fvars, mapping)
57.76 + module_name = modname(fvars)
57.77 +
57.78 + def reload_mapping():
57.79 + """loadhook to reload mapping and fvars."""
57.80 + mod = __import__(module_name, None, None, [''])
57.81 + mapping = getattr(mod, mapping_name, None)
57.82 + if mapping:
57.83 + self.fvars = mod.__dict__
57.84 + self.init_mapping(mapping)
57.85 +
57.86 + self.add_processor(loadhook(Reloader()))
57.87 + if mapping_name and module_name:
57.88 + self.add_processor(loadhook(reload_mapping))
57.89 +
57.90 + # load __main__ module usings its filename, so that it can be reloaded.
57.91 + if main_module_name() and '__main__' in sys.argv:
57.92 + try:
57.93 + __import__(main_module_name())
57.94 + except ImportError:
57.95 + pass
57.96 +
57.97 + def _load(self):
57.98 + web.ctx.app_stack.append(self)
57.99 +
57.100 + def _unload(self):
57.101 + web.ctx.app_stack = web.ctx.app_stack[:-1]
57.102 +
57.103 + if web.ctx.app_stack:
57.104 + # this is a sub-application, revert ctx to earlier state.
57.105 + oldctx = web.ctx.get('_oldctx')
57.106 + if oldctx:
57.107 + web.ctx.home = oldctx.home
57.108 + web.ctx.homepath = oldctx.homepath
57.109 + web.ctx.path = oldctx.path
57.110 + web.ctx.fullpath = oldctx.fullpath
57.111 +
57.112 + def _cleanup(self):
57.113 + # Threads can be recycled by WSGI servers.
57.114 + # Clearing up all thread-local state to avoid interefereing with subsequent requests.
57.115 + utils.ThreadedDict.clear_all()
57.116 +
57.117 + def init_mapping(self, mapping):
57.118 + self.mapping = list(utils.group(mapping, 2))
57.119 +
57.120 + def add_mapping(self, pattern, classname):
57.121 + self.mapping.append((pattern, classname))
57.122 +
57.123 + def add_processor(self, processor):
57.124 + """
57.125 + Adds a processor to the application.
57.126 +
57.127 + >>> urls = ("/(.*)", "echo")
57.128 + >>> app = application(urls, globals())
57.129 + >>> class echo:
57.130 + ... def GET(self, name): return name
57.131 + ...
57.132 + >>>
57.133 + >>> def hello(handler): return "hello, " + handler()
57.134 + ...
57.135 + >>> app.add_processor(hello)
57.136 + >>> app.request("/web.py").data
57.137 + 'hello, web.py'
57.138 + """
57.139 + self.processors.append(processor)
57.140 +
57.141 + def request(self, localpart='/', method='GET', data=None,
57.142 + host="0.0.0.0:8080", headers=None, https=False, **kw):
57.143 + """Makes request to this application for the specified path and method.
57.144 + Response will be a storage object with data, status and headers.
57.145 +
57.146 + >>> urls = ("/hello", "hello")
57.147 + >>> app = application(urls, globals())
57.148 + >>> class hello:
57.149 + ... def GET(self):
57.150 + ... web.header('Content-Type', 'text/plain')
57.151 + ... return "hello"
57.152 + ...
57.153 + >>> response = app.request("/hello")
57.154 + >>> response.data
57.155 + 'hello'
57.156 + >>> response.status
57.157 + '200 OK'
57.158 + >>> response.headers['Content-Type']
57.159 + 'text/plain'
57.160 +
57.161 + To use https, use https=True.
57.162 +
57.163 + >>> urls = ("/redirect", "redirect")
57.164 + >>> app = application(urls, globals())
57.165 + >>> class redirect:
57.166 + ... def GET(self): raise web.seeother("/foo")
57.167 + ...
57.168 + >>> response = app.request("/redirect")
57.169 + >>> response.headers['Location']
57.170 + 'http://0.0.0.0:8080/foo'
57.171 + >>> response = app.request("/redirect", https=True)
57.172 + >>> response.headers['Location']
57.173 + 'https://0.0.0.0:8080/foo'
57.174 +
57.175 + The headers argument specifies HTTP headers as a mapping object
57.176 + such as a dict.
57.177 +
57.178 + >>> urls = ('/ua', 'uaprinter')
57.179 + >>> class uaprinter:
57.180 + ... def GET(self):
57.181 + ... return 'your user-agent is ' + web.ctx.env['HTTP_USER_AGENT']
57.182 + ...
57.183 + >>> app = application(urls, globals())
57.184 + >>> app.request('/ua', headers = {
57.185 + ... 'User-Agent': 'a small jumping bean/1.0 (compatible)'
57.186 + ... }).data
57.187 + 'your user-agent is a small jumping bean/1.0 (compatible)'
57.188 +
57.189 + """
57.190 + path, maybe_query = urllib.splitquery(localpart)
57.191 + query = maybe_query or ""
57.192 +
57.193 + if 'env' in kw:
57.194 + env = kw['env']
57.195 + else:
57.196 + env = {}
57.197 + env = dict(env, HTTP_HOST=host, REQUEST_METHOD=method, PATH_INFO=path, QUERY_STRING=query, HTTPS=str(https))
57.198 + headers = headers or {}
57.199 +
57.200 + for k, v in headers.items():
57.201 + env['HTTP_' + k.upper().replace('-', '_')] = v
57.202 +
57.203 + if 'HTTP_CONTENT_LENGTH' in env:
57.204 + env['CONTENT_LENGTH'] = env.pop('HTTP_CONTENT_LENGTH')
57.205 +
57.206 + if 'HTTP_CONTENT_TYPE' in env:
57.207 + env['CONTENT_TYPE'] = env.pop('HTTP_CONTENT_TYPE')
57.208 +
57.209 + if method not in ["HEAD", "GET"]:
57.210 + data = data or ''
57.211 + import StringIO
57.212 + if isinstance(data, dict):
57.213 + q = urllib.urlencode(data)
57.214 + else:
57.215 + q = data
57.216 + env['wsgi.input'] = StringIO.StringIO(q)
57.217 + if not env.get('CONTENT_TYPE', '').lower().startswith('multipart/') and 'CONTENT_LENGTH' not in env:
57.218 + env['CONTENT_LENGTH'] = len(q)
57.219 + response = web.storage()
57.220 + def start_response(status, headers):
57.221 + response.status = status
57.222 + response.headers = dict(headers)
57.223 + response.header_items = headers
57.224 + response.data = "".join(self.wsgifunc()(env, start_response))
57.225 + return response
57.226 +
57.227 + def browser(self):
57.228 + import browser
57.229 + return browser.AppBrowser(self)
57.230 +
57.231 + def handle(self):
57.232 + fn, args = self._match(self.mapping, web.ctx.path)
57.233 + return self._delegate(fn, self.fvars, args)
57.234 +
57.235 + def handle_with_processors(self):
57.236 + def process(processors):
57.237 + try:
57.238 + if processors:
57.239 + p, processors = processors[0], processors[1:]
57.240 + return p(lambda: process(processors))
57.241 + else:
57.242 + return self.handle()
57.243 + except web.HTTPError:
57.244 + raise
57.245 + except (KeyboardInterrupt, SystemExit):
57.246 + raise
57.247 + except:
57.248 + print >> web.debug, traceback.format_exc()
57.249 + raise self.internalerror()
57.250 +
57.251 + # processors must be applied in the resvere order. (??)
57.252 + return process(self.processors)
57.253 +
57.254 + def wsgifunc(self, *middleware):
57.255 + """Returns a WSGI-compatible function for this application."""
57.256 + def peep(iterator):
57.257 + """Peeps into an iterator by doing an iteration
57.258 + and returns an equivalent iterator.
57.259 + """
57.260 + # wsgi requires the headers first
57.261 + # so we need to do an iteration
57.262 + # and save the result for later
57.263 + try:
57.264 + firstchunk = iterator.next()
57.265 + except StopIteration:
57.266 + firstchunk = ''
57.267 +
57.268 + return itertools.chain([firstchunk], iterator)
57.269 +
57.270 + def is_generator(x): return x and hasattr(x, 'next')
57.271 +
57.272 + def wsgi(env, start_resp):
57.273 + # clear threadlocal to avoid inteference of previous requests
57.274 + self._cleanup()
57.275 +
57.276 + self.load(env)
57.277 + try:
57.278 + # allow uppercase methods only
57.279 + if web.ctx.method.upper() != web.ctx.method:
57.280 + raise web.nomethod()
57.281 +
57.282 + result = self.handle_with_processors()
57.283 + if is_generator(result):
57.284 + result = peep(result)
57.285 + else:
57.286 + result = [result]
57.287 + except web.HTTPError, e:
57.288 + result = [e.data]
57.289 +
57.290 + result = web.safestr(iter(result))
57.291 +
57.292 + status, headers = web.ctx.status, web.ctx.headers
57.293 + start_resp(status, headers)
57.294 +
57.295 + def cleanup():
57.296 + self._cleanup()
57.297 + yield '' # force this function to be a generator
57.298 +
57.299 + return itertools.chain(result, cleanup())
57.300 +
57.301 + for m in middleware:
57.302 + wsgi = m(wsgi)
57.303 +
57.304 + return wsgi
57.305 +
57.306 + def run(self, *middleware):
57.307 + """
57.308 + Starts handling requests. If called in a CGI or FastCGI context, it will follow
57.309 + that protocol. If called from the command line, it will start an HTTP
57.310 + server on the port named in the first command line argument, or, if there
57.311 + is no argument, on port 8080.
57.312 +
57.313 + `middleware` is a list of WSGI middleware which is applied to the resulting WSGI
57.314 + function.
57.315 + """
57.316 + return wsgi.runwsgi(self.wsgifunc(*middleware))
57.317 +
57.318 + def stop(self):
57.319 + """Stops the http server started by run.
57.320 + """
57.321 + if httpserver.server:
57.322 + httpserver.server.stop()
57.323 + httpserver.server = None
57.324 +
57.325 + def cgirun(self, *middleware):
57.326 + """
57.327 + Return a CGI handler. This is mostly useful with Google App Engine.
57.328 + There you can just do:
57.329 +
57.330 + main = app.cgirun()
57.331 + """
57.332 + wsgiapp = self.wsgifunc(*middleware)
57.333 +
57.334 + try:
57.335 + from google.appengine.ext.webapp.util import run_wsgi_app
57.336 + return run_wsgi_app(wsgiapp)
57.337 + except ImportError:
57.338 + # we're not running from within Google App Engine
57.339 + return wsgiref.handlers.CGIHandler().run(wsgiapp)
57.340 +
57.341 + def load(self, env):
57.342 + """Initializes ctx using env."""
57.343 + ctx = web.ctx
57.344 + ctx.clear()
57.345 + ctx.status = '200 OK'
57.346 + ctx.headers = []
57.347 + ctx.output = ''
57.348 + ctx.environ = ctx.env = env
57.349 + ctx.host = env.get('HTTP_HOST')
57.350 +
57.351 + if env.get('wsgi.url_scheme') in ['http', 'https']:
57.352 + ctx.protocol = env['wsgi.url_scheme']
57.353 + elif env.get('HTTPS', '').lower() in ['on', 'true', '1']:
57.354 + ctx.protocol = 'https'
57.355 + else:
57.356 + ctx.protocol = 'http'
57.357 + ctx.homedomain = ctx.protocol + '://' + env.get('HTTP_HOST', '[unknown]')
57.358 + ctx.homepath = os.environ.get('REAL_SCRIPT_NAME', env.get('SCRIPT_NAME', ''))
57.359 + ctx.home = ctx.homedomain + ctx.homepath
57.360 + #@@ home is changed when the request is handled to a sub-application.
57.361 + #@@ but the real home is required for doing absolute redirects.
57.362 + ctx.realhome = ctx.home
57.363 + ctx.ip = env.get('REMOTE_ADDR')
57.364 + ctx.method = env.get('REQUEST_METHOD')
57.365 + ctx.path = env.get('PATH_INFO')
57.366 + # http://trac.lighttpd.net/trac/ticket/406 requires:
57.367 + if env.get('SERVER_SOFTWARE', '').startswith('lighttpd/'):
57.368 + ctx.path = lstrips(env.get('REQUEST_URI').split('?')[0], ctx.homepath)
57.369 + # Apache and CherryPy webservers unquote the url but lighttpd doesn't.
57.370 + # unquote explicitly for lighttpd to make ctx.path uniform across all servers.
57.371 + ctx.path = urllib.unquote(ctx.path)
57.372 +
57.373 + if env.get('QUERY_STRING'):
57.374 + ctx.query = '?' + env.get('QUERY_STRING', '')
57.375 + else:
57.376 + ctx.query = ''
57.377 +
57.378 + ctx.fullpath = ctx.path + ctx.query
57.379 +
57.380 + for k, v in ctx.iteritems():
57.381 + # convert all string values to unicode values and replace
57.382 + # malformed data with a suitable replacement marker.
57.383 + if isinstance(v, str):
57.384 + ctx[k] = v.decode('utf-8', 'replace')
57.385 +
57.386 + # status must always be str
57.387 + ctx.status = '200 OK'
57.388 +
57.389 + ctx.app_stack = []
57.390 +
57.391 + def _delegate(self, f, fvars, args=[]):
57.392 + def handle_class(cls):
57.393 + meth = web.ctx.method
57.394 + if meth == 'HEAD' and not hasattr(cls, meth):
57.395 + meth = 'GET'
57.396 + if not hasattr(cls, meth):
57.397 + raise web.nomethod(cls)
57.398 + tocall = getattr(cls(), meth)
57.399 + return tocall(*args)
57.400 +
57.401 + def is_class(o): return isinstance(o, (types.ClassType, type))
57.402 +
57.403 + if f is None:
57.404 + raise web.notfound()
57.405 + elif isinstance(f, application):
57.406 + return f.handle_with_processors()
57.407 + elif is_class(f):
57.408 + return handle_class(f)
57.409 + elif isinstance(f, basestring):
57.410 + if f.startswith('redirect '):
57.411 + url = f.split(' ', 1)[1]
57.412 + if web.ctx.method == "GET":
57.413 + x = web.ctx.env.get('QUERY_STRING', '')
57.414 + if x:
57.415 + url += '?' + x
57.416 + raise web.redirect(url)
57.417 + elif '.' in f:
57.418 + mod, cls = f.rsplit('.', 1)
57.419 + mod = __import__(mod, None, None, [''])
57.420 + cls = getattr(mod, cls)
57.421 + else:
57.422 + cls = fvars[f]
57.423 + return handle_class(cls)
57.424 + elif hasattr(f, '__call__'):
57.425 + return f()
57.426 + else:
57.427 + return web.notfound()
57.428 +
57.429 + def _match(self, mapping, value):
57.430 + for pat, what in mapping:
57.431 + if isinstance(what, application):
57.432 + if value.startswith(pat):
57.433 + f = lambda: self._delegate_sub_application(pat, what)
57.434 + return f, None
57.435 + else:
57.436 + continue
57.437 + elif isinstance(what, basestring):
57.438 + what, result = utils.re_subm('^' + pat + '$', what, value)
57.439 + else:
57.440 + result = utils.re_compile('^' + pat + '$').match(value)
57.441 +
57.442 + if result: # it's a match
57.443 + return what, [x for x in result.groups()]
57.444 + return None, None
57.445 +
57.446 + def _delegate_sub_application(self, dir, app):
57.447 + """Deletes request to sub application `app` rooted at the directory `dir`.
57.448 + The home, homepath, path and fullpath values in web.ctx are updated to mimic request
57.449 + to the subapp and are restored after it is handled.
57.450 +
57.451 + @@Any issues with when used with yield?
57.452 + """
57.453 + web.ctx._oldctx = web.storage(web.ctx)
57.454 + web.ctx.home += dir
57.455 + web.ctx.homepath += dir
57.456 + web.ctx.path = web.ctx.path[len(dir):]
57.457 + web.ctx.fullpath = web.ctx.fullpath[len(dir):]
57.458 + return app.handle_with_processors()
57.459 +
57.460 + def get_parent_app(self):
57.461 + if self in web.ctx.app_stack:
57.462 + index = web.ctx.app_stack.index(self)
57.463 + if index > 0:
57.464 + return web.ctx.app_stack[index-1]
57.465 +
57.466 + def notfound(self):
57.467 + """Returns HTTPError with '404 not found' message"""
57.468 + parent = self.get_parent_app()
57.469 + if parent:
57.470 + return parent.notfound()
57.471 + else:
57.472 + return web._NotFound()
57.473 +
57.474 + def internalerror(self):
57.475 + """Returns HTTPError with '500 internal error' message"""
57.476 + parent = self.get_parent_app()
57.477 + if parent:
57.478 + return parent.internalerror()
57.479 + elif web.config.get('debug'):
57.480 + import debugerror
57.481 + return debugerror.debugerror()
57.482 + else:
57.483 + return web._InternalError()
57.484 +
57.485 +class auto_application(application):
57.486 + """Application similar to `application` but urls are constructed
57.487 + automatiacally using metaclass.
57.488 +
57.489 + >>> app = auto_application()
57.490 + >>> class hello(app.page):
57.491 + ... def GET(self): return "hello, world"
57.492 + ...
57.493 + >>> class foo(app.page):
57.494 + ... path = '/foo/.*'
57.495 + ... def GET(self): return "foo"
57.496 + >>> app.request("/hello").data
57.497 + 'hello, world'
57.498 + >>> app.request('/foo/bar').data
57.499 + 'foo'
57.500 + """
57.501 + def __init__(self):
57.502 + application.__init__(self)
57.503 +
57.504 + class metapage(type):
57.505 + def __init__(klass, name, bases, attrs):
57.506 + type.__init__(klass, name, bases, attrs)
57.507 + path = attrs.get('path', '/' + name)
57.508 +
57.509 + # path can be specified as None to ignore that class
57.510 + # typically required to create a abstract base class.
57.511 + if path is not None:
57.512 + self.add_mapping(path, klass)
57.513 +
57.514 + class page:
57.515 + path = None
57.516 + __metaclass__ = metapage
57.517 +
57.518 + self.page = page
57.519 +
57.520 +# The application class already has the required functionality of subdir_application
57.521 +subdir_application = application
57.522 +
57.523 +class subdomain_application(application):
57.524 + """
57.525 + Application to delegate requests based on the host.
57.526 +
57.527 + >>> urls = ("/hello", "hello")
57.528 + >>> app = application(urls, globals())
57.529 + >>> class hello:
57.530 + ... def GET(self): return "hello"
57.531 + >>>
57.532 + >>> mapping = (r"hello\.example\.com", app)
57.533 + >>> app2 = subdomain_application(mapping)
57.534 + >>> app2.request("/hello", host="hello.example.com").data
57.535 + 'hello'
57.536 + >>> response = app2.request("/hello", host="something.example.com")
57.537 + >>> response.status
57.538 + '404 Not Found'
57.539 + >>> response.data
57.540 + 'not found'
57.541 + """
57.542 + def handle(self):
57.543 + host = web.ctx.host.split(':')[0] #strip port
57.544 + fn, args = self._match(self.mapping, host)
57.545 + return self._delegate(fn, self.fvars, args)
57.546 +
57.547 + def _match(self, mapping, value):
57.548 + for pat, what in mapping:
57.549 + if isinstance(what, basestring):
57.550 + what, result = utils.re_subm('^' + pat + '$', what, value)
57.551 + else:
57.552 + result = utils.re_compile('^' + pat + '$').match(value)
57.553 +
57.554 + if result: # it's a match
57.555 + return what, [x for x in result.groups()]
57.556 + return None, None
57.557 +
57.558 +def loadhook(h):
57.559 + """
57.560 + Converts a load hook into an application processor.
57.561 +
57.562 + >>> app = auto_application()
57.563 + >>> def f(): "something done before handling request"
57.564 + ...
57.565 + >>> app.add_processor(loadhook(f))
57.566 + """
57.567 + def processor(handler):
57.568 + h()
57.569 + return handler()
57.570 +
57.571 + return processor
57.572 +
57.573 +def unloadhook(h):
57.574 + """
57.575 + Converts an unload hook into an application processor.
57.576 +
57.577 + >>> app = auto_application()
57.578 + >>> def f(): "something done after handling request"
57.579 + ...
57.580 + >>> app.add_processor(unloadhook(f))
57.581 + """
57.582 + def processor(handler):
57.583 + try:
57.584 + result = handler()
57.585 + is_generator = result and hasattr(result, 'next')
57.586 + except:
57.587 + # run the hook even when handler raises some exception
57.588 + h()
57.589 + raise
57.590 +
57.591 + if is_generator:
57.592 + return wrap(result)
57.593 + else:
57.594 + h()
57.595 + return result
57.596 +
57.597 + def wrap(result):
57.598 + def next():
57.599 + try:
57.600 + return result.next()
57.601 + except:
57.602 + # call the hook at the and of iterator
57.603 + h()
57.604 + raise
57.605 +
57.606 + result = iter(result)
57.607 + while True:
57.608 + yield next()
57.609 +
57.610 + return processor
57.611 +
57.612 +def autodelegate(prefix=''):
57.613 + """
57.614 + Returns a method that takes one argument and calls the method named prefix+arg,
57.615 + calling `notfound()` if there isn't one. Example:
57.616 +
57.617 + urls = ('/prefs/(.*)', 'prefs')
57.618 +
57.619 + class prefs:
57.620 + GET = autodelegate('GET_')
57.621 + def GET_password(self): pass
57.622 + def GET_privacy(self): pass
57.623 +
57.624 + `GET_password` would get called for `/prefs/password` while `GET_privacy` for
57.625 + `GET_privacy` gets called for `/prefs/privacy`.
57.626 +
57.627 + If a user visits `/prefs/password/change` then `GET_password(self, '/change')`
57.628 + is called.
57.629 + """
57.630 + def internal(self, arg):
57.631 + if '/' in arg:
57.632 + first, rest = arg.split('/', 1)
57.633 + func = prefix + first
57.634 + args = ['/' + rest]
57.635 + else:
57.636 + func = prefix + arg
57.637 + args = []
57.638 +
57.639 + if hasattr(self, func):
57.640 + try:
57.641 + return getattr(self, func)(*args)
57.642 + except TypeError:
57.643 + raise web.notfound()
57.644 + else:
57.645 + raise web.notfound()
57.646 + return internal
57.647 +
57.648 +class Reloader:
57.649 + """Checks to see if any loaded modules have changed on disk and,
57.650 + if so, reloads them.
57.651 + """
57.652 +
57.653 + """File suffix of compiled modules."""
57.654 + if sys.platform.startswith('java'):
57.655 + SUFFIX = '$py.class'
57.656 + else:
57.657 + SUFFIX = '.pyc'
57.658 +
57.659 + def __init__(self):
57.660 + self.mtimes = {}
57.661 +
57.662 + def __call__(self):
57.663 + for mod in sys.modules.values():
57.664 + self.check(mod)
57.665 +
57.666 + def check(self, mod):
57.667 + # jython registers java packages as modules but they either
57.668 + # don't have a __file__ attribute or its value is None
57.669 + if not (mod and hasattr(mod, '__file__') and mod.__file__):
57.670 + return
57.671 +
57.672 + try:
57.673 + mtime = os.stat(mod.__file__).st_mtime
57.674 + except (OSError, IOError):
57.675 + return
57.676 + if mod.__file__.endswith(self.__class__.SUFFIX) and os.path.exists(mod.__file__[:-1]):
57.677 + mtime = max(os.stat(mod.__file__[:-1]).st_mtime, mtime)
57.678 +
57.679 + if mod not in self.mtimes:
57.680 + self.mtimes[mod] = mtime
57.681 + elif self.mtimes[mod] < mtime:
57.682 + try:
57.683 + reload(mod)
57.684 + self.mtimes[mod] = mtime
57.685 + except ImportError:
57.686 + pass
57.687 +
57.688 +if __name__ == "__main__":
57.689 + import doctest
57.690 + doctest.testmod()
58.1 Binary file OpenSecurity/install/web.py-0.37/web/application.pyc has changed
59.1 --- /dev/null Thu Jan 01 00:00:00 1970 +0000
59.2 +++ b/OpenSecurity/install/web.py-0.37/web/browser.py Mon Dec 02 14:02:05 2013 +0100
59.3 @@ -0,0 +1,236 @@
59.4 +"""Browser to test web applications.
59.5 +(from web.py)
59.6 +"""
59.7 +from utils import re_compile
59.8 +from net import htmlunquote
59.9 +
59.10 +import httplib, urllib, urllib2
59.11 +import copy
59.12 +from StringIO import StringIO
59.13 +
59.14 +DEBUG = False
59.15 +
59.16 +__all__ = [
59.17 + "BrowserError",
59.18 + "Browser", "AppBrowser",
59.19 + "AppHandler"
59.20 +]
59.21 +
59.22 +class BrowserError(Exception):
59.23 + pass
59.24 +
59.25 +class Browser:
59.26 + def __init__(self):
59.27 + import cookielib
59.28 + self.cookiejar = cookielib.CookieJar()
59.29 + self._cookie_processor = urllib2.HTTPCookieProcessor(self.cookiejar)
59.30 + self.form = None
59.31 +
59.32 + self.url = "http://0.0.0.0:8080/"
59.33 + self.path = "/"
59.34 +
59.35 + self.status = None
59.36 + self.data = None
59.37 + self._response = None
59.38 + self._forms = None
59.39 +
59.40 + def reset(self):
59.41 + """Clears all cookies and history."""
59.42 + self.cookiejar.clear()
59.43 +
59.44 + def build_opener(self):
59.45 + """Builds the opener using urllib2.build_opener.
59.46 + Subclasses can override this function to prodive custom openers.
59.47 + """
59.48 + return urllib2.build_opener()
59.49 +
59.50 + def do_request(self, req):
59.51 + if DEBUG:
59.52 + print 'requesting', req.get_method(), req.get_full_url()
59.53 + opener = self.build_opener()
59.54 + opener.add_handler(self._cookie_processor)
59.55 + try:
59.56 + self._response = opener.open(req)
59.57 + except urllib2.HTTPError, e:
59.58 + self._response = e
59.59 +
59.60 + self.url = self._response.geturl()
59.61 + self.path = urllib2.Request(self.url).get_selector()
59.62 + self.data = self._response.read()
59.63 + self.status = self._response.code
59.64 + self._forms = None
59.65 + self.form = None
59.66 + return self.get_response()
59.67 +
59.68 + def open(self, url, data=None, headers={}):
59.69 + """Opens the specified url."""
59.70 + url = urllib.basejoin(self.url, url)
59.71 + req = urllib2.Request(url, data, headers)
59.72 + return self.do_request(req)
59.73 +
59.74 + def show(self):
59.75 + """Opens the current page in real web browser."""
59.76 + f = open('page.html', 'w')
59.77 + f.write(self.data)
59.78 + f.close()
59.79 +
59.80 + import webbrowser, os
59.81 + url = 'file://' + os.path.abspath('page.html')
59.82 + webbrowser.open(url)
59.83 +
59.84 + def get_response(self):
59.85 + """Returns a copy of the current response."""
59.86 + return urllib.addinfourl(StringIO(self.data), self._response.info(), self._response.geturl())
59.87 +
59.88 + def get_soup(self):
59.89 + """Returns beautiful soup of the current document."""
59.90 + import BeautifulSoup
59.91 + return BeautifulSoup.BeautifulSoup(self.data)
59.92 +
59.93 + def get_text(self, e=None):
59.94 + """Returns content of e or the current document as plain text."""
59.95 + e = e or self.get_soup()
59.96 + return ''.join([htmlunquote(c) for c in e.recursiveChildGenerator() if isinstance(c, unicode)])
59.97 +
59.98 + def _get_links(self):
59.99 + soup = self.get_soup()
59.100 + return [a for a in soup.findAll(name='a')]
59.101 +
59.102 + def get_links(self, text=None, text_regex=None, url=None, url_regex=None, predicate=None):
59.103 + """Returns all links in the document."""
59.104 + return self._filter_links(self._get_links(),
59.105 + text=text, text_regex=text_regex, url=url, url_regex=url_regex, predicate=predicate)
59.106 +
59.107 + def follow_link(self, link=None, text=None, text_regex=None, url=None, url_regex=None, predicate=None):
59.108 + if link is None:
59.109 + links = self._filter_links(self.get_links(),
59.110 + text=text, text_regex=text_regex, url=url, url_regex=url_regex, predicate=predicate)
59.111 + link = links and links[0]
59.112 +
59.113 + if link:
59.114 + return self.open(link['href'])
59.115 + else:
59.116 + raise BrowserError("No link found")
59.117 +
59.118 + def find_link(self, text=None, text_regex=None, url=None, url_regex=None, predicate=None):
59.119 + links = self._filter_links(self.get_links(),
59.120 + text=text, text_regex=text_regex, url=url, url_regex=url_regex, predicate=predicate)
59.121 + return links and links[0] or None
59.122 +
59.123 + def _filter_links(self, links,
59.124 + text=None, text_regex=None,
59.125 + url=None, url_regex=None,
59.126 + predicate=None):
59.127 + predicates = []
59.128 + if text is not None:
59.129 + predicates.append(lambda link: link.string == text)
59.130 + if text_regex is not None:
59.131 + predicates.append(lambda link: re_compile(text_regex).search(link.string or ''))
59.132 + if url is not None:
59.133 + predicates.append(lambda link: link.get('href') == url)
59.134 + if url_regex is not None:
59.135 + predicates.append(lambda link: re_compile(url_regex).search(link.get('href', '')))
59.136 + if predicate:
59.137 + predicate.append(predicate)
59.138 +
59.139 + def f(link):
59.140 + for p in predicates:
59.141 + if not p(link):
59.142 + return False
59.143 + return True
59.144 +
59.145 + return [link for link in links if f(link)]
59.146 +
59.147 + def get_forms(self):
59.148 + """Returns all forms in the current document.
59.149 + The returned form objects implement the ClientForm.HTMLForm interface.
59.150 + """
59.151 + if self._forms is None:
59.152 + import ClientForm
59.153 + self._forms = ClientForm.ParseResponse(self.get_response(), backwards_compat=False)
59.154 + return self._forms
59.155 +
59.156 + def select_form(self, name=None, predicate=None, index=0):
59.157 + """Selects the specified form."""
59.158 + forms = self.get_forms()
59.159 +
59.160 + if name is not None:
59.161 + forms = [f for f in forms if f.name == name]
59.162 + if predicate:
59.163 + forms = [f for f in forms if predicate(f)]
59.164 +
59.165 + if forms:
59.166 + self.form = forms[index]
59.167 + return self.form
59.168 + else:
59.169 + raise BrowserError("No form selected.")
59.170 +
59.171 + def submit(self, **kw):
59.172 + """submits the currently selected form."""
59.173 + if self.form is None:
59.174 + raise BrowserError("No form selected.")
59.175 + req = self.form.click(**kw)
59.176 + return self.do_request(req)
59.177 +
59.178 + def __getitem__(self, key):
59.179 + return self.form[key]
59.180 +
59.181 + def __setitem__(self, key, value):
59.182 + self.form[key] = value
59.183 +
59.184 +class AppBrowser(Browser):
59.185 + """Browser interface to test web.py apps.
59.186 +
59.187 + b = AppBrowser(app)
59.188 + b.open('/')
59.189 + b.follow_link(text='Login')
59.190 +
59.191 + b.select_form(name='login')
59.192 + b['username'] = 'joe'
59.193 + b['password'] = 'secret'
59.194 + b.submit()
59.195 +
59.196 + assert b.path == '/'
59.197 + assert 'Welcome joe' in b.get_text()
59.198 + """
59.199 + def __init__(self, app):
59.200 + Browser.__init__(self)
59.201 + self.app = app
59.202 +
59.203 + def build_opener(self):
59.204 + return urllib2.build_opener(AppHandler(self.app))
59.205 +
59.206 +class AppHandler(urllib2.HTTPHandler):
59.207 + """urllib2 handler to handle requests using web.py application."""
59.208 + handler_order = 100
59.209 +
59.210 + def __init__(self, app):
59.211 + self.app = app
59.212 +
59.213 + def http_open(self, req):
59.214 + result = self.app.request(
59.215 + localpart=req.get_selector(),
59.216 + method=req.get_method(),
59.217 + host=req.get_host(),
59.218 + data=req.get_data(),
59.219 + headers=dict(req.header_items()),
59.220 + https=req.get_type() == "https"
59.221 + )
59.222 + return self._make_response(result, req.get_full_url())
59.223 +
59.224 + def https_open(self, req):
59.225 + return self.http_open(req)
59.226 +
59.227 + try:
59.228 + https_request = urllib2.HTTPHandler.do_request_
59.229 + except AttributeError:
59.230 + # for python 2.3
59.231 + pass
59.232 +
59.233 + def _make_response(self, result, url):
59.234 + data = "\r\n".join(["%s: %s" % (k, v) for k, v in result.header_items])
59.235 + headers = httplib.HTTPMessage(StringIO(data))
59.236 + response = urllib.addinfourl(StringIO(result.data), headers, url)
59.237 + code, msg = result.status.split(None, 1)
59.238 + response.code, response.msg = int(code), msg
59.239 + return response
60.1 Binary file OpenSecurity/install/web.py-0.37/web/browser.pyc has changed
61.1 --- /dev/null Thu Jan 01 00:00:00 1970 +0000
61.2 +++ b/OpenSecurity/install/web.py-0.37/web/contrib/template.py Mon Dec 02 14:02:05 2013 +0100
61.3 @@ -0,0 +1,131 @@
61.4 +"""
61.5 +Interface to various templating engines.
61.6 +"""
61.7 +import os.path
61.8 +
61.9 +__all__ = [
61.10 + "render_cheetah", "render_genshi", "render_mako",
61.11 + "cache",
61.12 +]
61.13 +
61.14 +class render_cheetah:
61.15 + """Rendering interface to Cheetah Templates.
61.16 +
61.17 + Example:
61.18 +
61.19 + render = render_cheetah('templates')
61.20 + render.hello(name="cheetah")
61.21 + """
61.22 + def __init__(self, path):
61.23 + # give error if Chetah is not installed
61.24 + from Cheetah.Template import Template
61.25 + self.path = path
61.26 +
61.27 + def __getattr__(self, name):
61.28 + from Cheetah.Template import Template
61.29 + path = os.path.join(self.path, name + ".html")
61.30 +
61.31 + def template(**kw):
61.32 + t = Template(file=path, searchList=[kw])
61.33 + return t.respond()
61.34 +
61.35 + return template
61.36 +
61.37 +class render_genshi:
61.38 + """Rendering interface genshi templates.
61.39 + Example:
61.40 +
61.41 + for xml/html templates.
61.42 +
61.43 + render = render_genshi(['templates/'])
61.44 + render.hello(name='genshi')
61.45 +
61.46 + For text templates:
61.47 +
61.48 + render = render_genshi(['templates/'], type='text')
61.49 + render.hello(name='genshi')
61.50 + """
61.51 +
61.52 + def __init__(self, *a, **kwargs):
61.53 + from genshi.template import TemplateLoader
61.54 +
61.55 + self._type = kwargs.pop('type', None)
61.56 + self._loader = TemplateLoader(*a, **kwargs)
61.57 +
61.58 + def __getattr__(self, name):
61.59 + # Assuming all templates are html
61.60 + path = name + ".html"
61.61 +
61.62 + if self._type == "text":
61.63 + from genshi.template import TextTemplate
61.64 + cls = TextTemplate
61.65 + type = "text"
61.66 + else:
61.67 + cls = None
61.68 + type = None
61.69 +
61.70 + t = self._loader.load(path, cls=cls)
61.71 + def template(**kw):
61.72 + stream = t.generate(**kw)
61.73 + if type:
61.74 + return stream.render(type)
61.75 + else:
61.76 + return stream.render()
61.77 + return template
61.78 +
61.79 +class render_jinja:
61.80 + """Rendering interface to Jinja2 Templates
61.81 +
61.82 + Example:
61.83 +
61.84 + render= render_jinja('templates')
61.85 + render.hello(name='jinja2')
61.86 + """
61.87 + def __init__(self, *a, **kwargs):
61.88 + extensions = kwargs.pop('extensions', [])
61.89 + globals = kwargs.pop('globals', {})
61.90 +
61.91 + from jinja2 import Environment,FileSystemLoader
61.92 + self._lookup = Environment(loader=FileSystemLoader(*a, **kwargs), extensions=extensions)
61.93 + self._lookup.globals.update(globals)
61.94 +
61.95 + def __getattr__(self, name):
61.96 + # Assuming all templates end with .html
61.97 + path = name + '.html'
61.98 + t = self._lookup.get_template(path)
61.99 + return t.render
61.100 +
61.101 +class render_mako:
61.102 + """Rendering interface to Mako Templates.
61.103 +
61.104 + Example:
61.105 +
61.106 + render = render_mako(directories=['templates'])
61.107 + render.hello(name="mako")
61.108 + """
61.109 + def __init__(self, *a, **kwargs):
61.110 + from mako.lookup import TemplateLookup
61.111 + self._lookup = TemplateLookup(*a, **kwargs)
61.112 +
61.113 + def __getattr__(self, name):
61.114 + # Assuming all templates are html
61.115 + path = name + ".html"
61.116 + t = self._lookup.get_template(path)
61.117 + return t.render
61.118 +
61.119 +class cache:
61.120 + """Cache for any rendering interface.
61.121 +
61.122 + Example:
61.123 +
61.124 + render = cache(render_cheetah("templates/"))
61.125 + render.hello(name='cache')
61.126 + """
61.127 + def __init__(self, render):
61.128 + self._render = render
61.129 + self._cache = {}
61.130 +
61.131 + def __getattr__(self, name):
61.132 + if name not in self._cache:
61.133 + self._cache[name] = getattr(self._render, name)
61.134 + return self._cache[name]
62.1 --- /dev/null Thu Jan 01 00:00:00 1970 +0000
62.2 +++ b/OpenSecurity/install/web.py-0.37/web/db.py Mon Dec 02 14:02:05 2013 +0100
62.3 @@ -0,0 +1,1237 @@
62.4 +"""
62.5 +Database API
62.6 +(part of web.py)
62.7 +"""
62.8 +
62.9 +__all__ = [
62.10 + "UnknownParamstyle", "UnknownDB", "TransactionError",
62.11 + "sqllist", "sqlors", "reparam", "sqlquote",
62.12 + "SQLQuery", "SQLParam", "sqlparam",
62.13 + "SQLLiteral", "sqlliteral",
62.14 + "database", 'DB',
62.15 +]
62.16 +
62.17 +import time
62.18 +try:
62.19 + import datetime
62.20 +except ImportError:
62.21 + datetime = None
62.22 +
62.23 +try: set
62.24 +except NameError:
62.25 + from sets import Set as set
62.26 +
62.27 +from utils import threadeddict, storage, iters, iterbetter, safestr, safeunicode
62.28 +
62.29 +try:
62.30 + # db module can work independent of web.py
62.31 + from webapi import debug, config
62.32 +except:
62.33 + import sys
62.34 + debug = sys.stderr
62.35 + config = storage()
62.36 +
62.37 +class UnknownDB(Exception):
62.38 + """raised for unsupported dbms"""
62.39 + pass
62.40 +
62.41 +class _ItplError(ValueError):
62.42 + def __init__(self, text, pos):
62.43 + ValueError.__init__(self)
62.44 + self.text = text
62.45 + self.pos = pos
62.46 + def __str__(self):
62.47 + return "unfinished expression in %s at char %d" % (
62.48 + repr(self.text), self.pos)
62.49 +
62.50 +class TransactionError(Exception): pass
62.51 +
62.52 +class UnknownParamstyle(Exception):
62.53 + """
62.54 + raised for unsupported db paramstyles
62.55 +
62.56 + (currently supported: qmark, numeric, format, pyformat)
62.57 + """
62.58 + pass
62.59 +
62.60 +class SQLParam(object):
62.61 + """
62.62 + Parameter in SQLQuery.
62.63 +
62.64 + >>> q = SQLQuery(["SELECT * FROM test WHERE name=", SQLParam("joe")])
62.65 + >>> q
62.66 + <sql: "SELECT * FROM test WHERE name='joe'">
62.67 + >>> q.query()
62.68 + 'SELECT * FROM test WHERE name=%s'
62.69 + >>> q.values()
62.70 + ['joe']
62.71 + """
62.72 + __slots__ = ["value"]
62.73 +
62.74 + def __init__(self, value):
62.75 + self.value = value
62.76 +
62.77 + def get_marker(self, paramstyle='pyformat'):
62.78 + if paramstyle == 'qmark':
62.79 + return '?'
62.80 + elif paramstyle == 'numeric':
62.81 + return ':1'
62.82 + elif paramstyle is None or paramstyle in ['format', 'pyformat']:
62.83 + return '%s'
62.84 + raise UnknownParamstyle, paramstyle
62.85 +
62.86 + def sqlquery(self):
62.87 + return SQLQuery([self])
62.88 +
62.89 + def __add__(self, other):
62.90 + return self.sqlquery() + other
62.91 +
62.92 + def __radd__(self, other):
62.93 + return other + self.sqlquery()
62.94 +
62.95 + def __str__(self):
62.96 + return str(self.value)
62.97 +
62.98 + def __repr__(self):
62.99 + return '<param: %s>' % repr(self.value)
62.100 +
62.101 +sqlparam = SQLParam
62.102 +
62.103 +class SQLQuery(object):
62.104 + """
62.105 + You can pass this sort of thing as a clause in any db function.
62.106 + Otherwise, you can pass a dictionary to the keyword argument `vars`
62.107 + and the function will call reparam for you.
62.108 +
62.109 + Internally, consists of `items`, which is a list of strings and
62.110 + SQLParams, which get concatenated to produce the actual query.
62.111 + """
62.112 + __slots__ = ["items"]
62.113 +
62.114 + # tested in sqlquote's docstring
62.115 + def __init__(self, items=None):
62.116 + r"""Creates a new SQLQuery.
62.117 +
62.118 + >>> SQLQuery("x")
62.119 + <sql: 'x'>
62.120 + >>> q = SQLQuery(['SELECT * FROM ', 'test', ' WHERE x=', SQLParam(1)])
62.121 + >>> q
62.122 + <sql: 'SELECT * FROM test WHERE x=1'>
62.123 + >>> q.query(), q.values()
62.124 + ('SELECT * FROM test WHERE x=%s', [1])
62.125 + >>> SQLQuery(SQLParam(1))
62.126 + <sql: '1'>
62.127 + """
62.128 + if items is None:
62.129 + self.items = []
62.130 + elif isinstance(items, list):
62.131 + self.items = items
62.132 + elif isinstance(items, SQLParam):
62.133 + self.items = [items]
62.134 + elif isinstance(items, SQLQuery):
62.135 + self.items = list(items.items)
62.136 + else:
62.137 + self.items = [items]
62.138 +
62.139 + # Take care of SQLLiterals
62.140 + for i, item in enumerate(self.items):
62.141 + if isinstance(item, SQLParam) and isinstance(item.value, SQLLiteral):
62.142 + self.items[i] = item.value.v
62.143 +
62.144 + def append(self, value):
62.145 + self.items.append(value)
62.146 +
62.147 + def __add__(self, other):
62.148 + if isinstance(other, basestring):
62.149 + items = [other]
62.150 + elif isinstance(other, SQLQuery):
62.151 + items = other.items
62.152 + else:
62.153 + return NotImplemented
62.154 + return SQLQuery(self.items + items)
62.155 +
62.156 + def __radd__(self, other):
62.157 + if isinstance(other, basestring):
62.158 + items = [other]
62.159 + else:
62.160 + return NotImplemented
62.161 +
62.162 + return SQLQuery(items + self.items)
62.163 +
62.164 + def __iadd__(self, other):
62.165 + if isinstance(other, (basestring, SQLParam)):
62.166 + self.items.append(other)
62.167 + elif isinstance(other, SQLQuery):
62.168 + self.items.extend(other.items)
62.169 + else:
62.170 + return NotImplemented
62.171 + return self
62.172 +
62.173 + def __len__(self):
62.174 + return len(self.query())
62.175 +
62.176 + def query(self, paramstyle=None):
62.177 + """
62.178 + Returns the query part of the sql query.
62.179 + >>> q = SQLQuery(["SELECT * FROM test WHERE name=", SQLParam('joe')])
62.180 + >>> q.query()
62.181 + 'SELECT * FROM test WHERE name=%s'
62.182 + >>> q.query(paramstyle='qmark')
62.183 + 'SELECT * FROM test WHERE name=?'
62.184 + """
62.185 + s = []
62.186 + for x in self.items:
62.187 + if isinstance(x, SQLParam):
62.188 + x = x.get_marker(paramstyle)
62.189 + s.append(safestr(x))
62.190 + else:
62.191 + x = safestr(x)
62.192 + # automatically escape % characters in the query
62.193 + # For backward compatability, ignore escaping when the query looks already escaped
62.194 + if paramstyle in ['format', 'pyformat']:
62.195 + if '%' in x and '%%' not in x:
62.196 + x = x.replace('%', '%%')
62.197 + s.append(x)
62.198 + return "".join(s)
62.199 +
62.200 + def values(self):
62.201 + """
62.202 + Returns the values of the parameters used in the sql query.
62.203 + >>> q = SQLQuery(["SELECT * FROM test WHERE name=", SQLParam('joe')])
62.204 + >>> q.values()
62.205 + ['joe']
62.206 + """
62.207 + return [i.value for i in self.items if isinstance(i, SQLParam)]
62.208 +
62.209 + def join(items, sep=' ', prefix=None, suffix=None, target=None):
62.210 + """
62.211 + Joins multiple queries.
62.212 +
62.213 + >>> SQLQuery.join(['a', 'b'], ', ')
62.214 + <sql: 'a, b'>
62.215 +
62.216 + Optinally, prefix and suffix arguments can be provided.
62.217 +
62.218 + >>> SQLQuery.join(['a', 'b'], ', ', prefix='(', suffix=')')
62.219 + <sql: '(a, b)'>
62.220 +
62.221 + If target argument is provided, the items are appended to target instead of creating a new SQLQuery.
62.222 + """
62.223 + if target is None:
62.224 + target = SQLQuery()
62.225 +
62.226 + target_items = target.items
62.227 +
62.228 + if prefix:
62.229 + target_items.append(prefix)
62.230 +
62.231 + for i, item in enumerate(items):
62.232 + if i != 0:
62.233 + target_items.append(sep)
62.234 + if isinstance(item, SQLQuery):
62.235 + target_items.extend(item.items)
62.236 + else:
62.237 + target_items.append(item)
62.238 +
62.239 + if suffix:
62.240 + target_items.append(suffix)
62.241 + return target
62.242 +
62.243 + join = staticmethod(join)
62.244 +
62.245 + def _str(self):
62.246 + try:
62.247 + return self.query() % tuple([sqlify(x) for x in self.values()])
62.248 + except (ValueError, TypeError):
62.249 + return self.query()
62.250 +
62.251 + def __str__(self):
62.252 + return safestr(self._str())
62.253 +
62.254 + def __unicode__(self):
62.255 + return safeunicode(self._str())
62.256 +
62.257 + def __repr__(self):
62.258 + return '<sql: %s>' % repr(str(self))
62.259 +
62.260 +class SQLLiteral:
62.261 + """
62.262 + Protects a string from `sqlquote`.
62.263 +
62.264 + >>> sqlquote('NOW()')
62.265 + <sql: "'NOW()'">
62.266 + >>> sqlquote(SQLLiteral('NOW()'))
62.267 + <sql: 'NOW()'>
62.268 + """
62.269 + def __init__(self, v):
62.270 + self.v = v
62.271 +
62.272 + def __repr__(self):
62.273 + return self.v
62.274 +
62.275 +sqlliteral = SQLLiteral
62.276 +
62.277 +def _sqllist(values):
62.278 + """
62.279 + >>> _sqllist([1, 2, 3])
62.280 + <sql: '(1, 2, 3)'>
62.281 + """
62.282 + items = []
62.283 + items.append('(')
62.284 + for i, v in enumerate(values):
62.285 + if i != 0:
62.286 + items.append(', ')
62.287 + items.append(sqlparam(v))
62.288 + items.append(')')
62.289 + return SQLQuery(items)
62.290 +
62.291 +def reparam(string_, dictionary):
62.292 + """
62.293 + Takes a string and a dictionary and interpolates the string
62.294 + using values from the dictionary. Returns an `SQLQuery` for the result.
62.295 +
62.296 + >>> reparam("s = $s", dict(s=True))
62.297 + <sql: "s = 't'">
62.298 + >>> reparam("s IN $s", dict(s=[1, 2]))
62.299 + <sql: 's IN (1, 2)'>
62.300 + """
62.301 + dictionary = dictionary.copy() # eval mucks with it
62.302 + vals = []
62.303 + result = []
62.304 + for live, chunk in _interpolate(string_):
62.305 + if live:
62.306 + v = eval(chunk, dictionary)
62.307 + result.append(sqlquote(v))
62.308 + else:
62.309 + result.append(chunk)
62.310 + return SQLQuery.join(result, '')
62.311 +
62.312 +def sqlify(obj):
62.313 + """
62.314 + converts `obj` to its proper SQL version
62.315 +
62.316 + >>> sqlify(None)
62.317 + 'NULL'
62.318 + >>> sqlify(True)
62.319 + "'t'"
62.320 + >>> sqlify(3)
62.321 + '3'
62.322 + """
62.323 + # because `1 == True and hash(1) == hash(True)`
62.324 + # we have to do this the hard way...
62.325 +
62.326 + if obj is None:
62.327 + return 'NULL'
62.328 + elif obj is True:
62.329 + return "'t'"
62.330 + elif obj is False:
62.331 + return "'f'"
62.332 + elif datetime and isinstance(obj, datetime.datetime):
62.333 + return repr(obj.isoformat())
62.334 + else:
62.335 + if isinstance(obj, unicode): obj = obj.encode('utf8')
62.336 + return repr(obj)
62.337 +
62.338 +def sqllist(lst):
62.339 + """
62.340 + Converts the arguments for use in something like a WHERE clause.
62.341 +
62.342 + >>> sqllist(['a', 'b'])
62.343 + 'a, b'
62.344 + >>> sqllist('a')
62.345 + 'a'
62.346 + >>> sqllist(u'abc')
62.347 + u'abc'
62.348 + """
62.349 + if isinstance(lst, basestring):
62.350 + return lst
62.351 + else:
62.352 + return ', '.join(lst)
62.353 +
62.354 +def sqlors(left, lst):
62.355 + """
62.356 + `left is a SQL clause like `tablename.arg = `
62.357 + and `lst` is a list of values. Returns a reparam-style
62.358 + pair featuring the SQL that ORs together the clause
62.359 + for each item in the lst.
62.360 +
62.361 + >>> sqlors('foo = ', [])
62.362 + <sql: '1=2'>
62.363 + >>> sqlors('foo = ', [1])
62.364 + <sql: 'foo = 1'>
62.365 + >>> sqlors('foo = ', 1)
62.366 + <sql: 'foo = 1'>
62.367 + >>> sqlors('foo = ', [1,2,3])
62.368 + <sql: '(foo = 1 OR foo = 2 OR foo = 3 OR 1=2)'>
62.369 + """
62.370 + if isinstance(lst, iters):
62.371 + lst = list(lst)
62.372 + ln = len(lst)
62.373 + if ln == 0:
62.374 + return SQLQuery("1=2")
62.375 + if ln == 1:
62.376 + lst = lst[0]
62.377 +
62.378 + if isinstance(lst, iters):
62.379 + return SQLQuery(['('] +
62.380 + sum([[left, sqlparam(x), ' OR '] for x in lst], []) +
62.381 + ['1=2)']
62.382 + )
62.383 + else:
62.384 + return left + sqlparam(lst)
62.385 +
62.386 +def sqlwhere(dictionary, grouping=' AND '):
62.387 + """
62.388 + Converts a `dictionary` to an SQL WHERE clause `SQLQuery`.
62.389 +
62.390 + >>> sqlwhere({'cust_id': 2, 'order_id':3})
62.391 + <sql: 'order_id = 3 AND cust_id = 2'>
62.392 + >>> sqlwhere({'cust_id': 2, 'order_id':3}, grouping=', ')
62.393 + <sql: 'order_id = 3, cust_id = 2'>
62.394 + >>> sqlwhere({'a': 'a', 'b': 'b'}).query()
62.395 + 'a = %s AND b = %s'
62.396 + """
62.397 + return SQLQuery.join([k + ' = ' + sqlparam(v) for k, v in dictionary.items()], grouping)
62.398 +
62.399 +def sqlquote(a):
62.400 + """
62.401 + Ensures `a` is quoted properly for use in a SQL query.
62.402 +
62.403 + >>> 'WHERE x = ' + sqlquote(True) + ' AND y = ' + sqlquote(3)
62.404 + <sql: "WHERE x = 't' AND y = 3">
62.405 + >>> 'WHERE x = ' + sqlquote(True) + ' AND y IN ' + sqlquote([2, 3])
62.406 + <sql: "WHERE x = 't' AND y IN (2, 3)">
62.407 + """
62.408 + if isinstance(a, list):
62.409 + return _sqllist(a)
62.410 + else:
62.411 + return sqlparam(a).sqlquery()
62.412 +
62.413 +class Transaction:
62.414 + """Database transaction."""
62.415 + def __init__(self, ctx):
62.416 + self.ctx = ctx
62.417 + self.transaction_count = transaction_count = len(ctx.transactions)
62.418 +
62.419 + class transaction_engine:
62.420 + """Transaction Engine used in top level transactions."""
62.421 + def do_transact(self):
62.422 + ctx.commit(unload=False)
62.423 +
62.424 + def do_commit(self):
62.425 + ctx.commit()
62.426 +
62.427 + def do_rollback(self):
62.428 + ctx.rollback()
62.429 +
62.430 + class subtransaction_engine:
62.431 + """Transaction Engine used in sub transactions."""
62.432 + def query(self, q):
62.433 + db_cursor = ctx.db.cursor()
62.434 + ctx.db_execute(db_cursor, SQLQuery(q % transaction_count))
62.435 +
62.436 + def do_transact(self):
62.437 + self.query('SAVEPOINT webpy_sp_%s')
62.438 +
62.439 + def do_commit(self):
62.440 + self.query('RELEASE SAVEPOINT webpy_sp_%s')
62.441 +
62.442 + def do_rollback(self):
62.443 + self.query('ROLLBACK TO SAVEPOINT webpy_sp_%s')
62.444 +
62.445 + class dummy_engine:
62.446 + """Transaction Engine used instead of subtransaction_engine
62.447 + when sub transactions are not supported."""
62.448 + do_transact = do_commit = do_rollback = lambda self: None
62.449 +
62.450 + if self.transaction_count:
62.451 + # nested transactions are not supported in some databases
62.452 + if self.ctx.get('ignore_nested_transactions'):
62.453 + self.engine = dummy_engine()
62.454 + else:
62.455 + self.engine = subtransaction_engine()
62.456 + else:
62.457 + self.engine = transaction_engine()
62.458 +
62.459 + self.engine.do_transact()
62.460 + self.ctx.transactions.append(self)
62.461 +
62.462 + def __enter__(self):
62.463 + return self
62.464 +
62.465 + def __exit__(self, exctype, excvalue, traceback):
62.466 + if exctype is not None:
62.467 + self.rollback()
62.468 + else:
62.469 + self.commit()
62.470 +
62.471 + def commit(self):
62.472 + if len(self.ctx.transactions) > self.transaction_count:
62.473 + self.engine.do_commit()
62.474 + self.ctx.transactions = self.ctx.transactions[:self.transaction_count]
62.475 +
62.476 + def rollback(self):
62.477 + if len(self.ctx.transactions) > self.transaction_count:
62.478 + self.engine.do_rollback()
62.479 + self.ctx.transactions = self.ctx.transactions[:self.transaction_count]
62.480 +
62.481 +class DB:
62.482 + """Database"""
62.483 + def __init__(self, db_module, keywords):
62.484 + """Creates a database.
62.485 + """
62.486 + # some DB implementaions take optional paramater `driver` to use a specific driver modue
62.487 + # but it should not be passed to connect
62.488 + keywords.pop('driver', None)
62.489 +
62.490 + self.db_module = db_module
62.491 + self.keywords = keywords
62.492 +
62.493 + self._ctx = threadeddict()
62.494 + # flag to enable/disable printing queries
62.495 + self.printing = config.get('debug_sql', config.get('debug', False))
62.496 + self.supports_multiple_insert = False
62.497 +
62.498 + try:
62.499 + import DBUtils
62.500 + # enable pooling if DBUtils module is available.
62.501 + self.has_pooling = True
62.502 + except ImportError:
62.503 + self.has_pooling = False
62.504 +
62.505 + # Pooling can be disabled by passing pooling=False in the keywords.
62.506 + self.has_pooling = self.keywords.pop('pooling', True) and self.has_pooling
62.507 +
62.508 + def _getctx(self):
62.509 + if not self._ctx.get('db'):
62.510 + self._load_context(self._ctx)
62.511 + return self._ctx
62.512 + ctx = property(_getctx)
62.513 +
62.514 + def _load_context(self, ctx):
62.515 + ctx.dbq_count = 0
62.516 + ctx.transactions = [] # stack of transactions
62.517 +
62.518 + if self.has_pooling:
62.519 + ctx.db = self._connect_with_pooling(self.keywords)
62.520 + else:
62.521 + ctx.db = self._connect(self.keywords)
62.522 + ctx.db_execute = self._db_execute
62.523 +
62.524 + if not hasattr(ctx.db, 'commit'):
62.525 + ctx.db.commit = lambda: None
62.526 +
62.527 + if not hasattr(ctx.db, 'rollback'):
62.528 + ctx.db.rollback = lambda: None
62.529 +
62.530 + def commit(unload=True):
62.531 + # do db commit and release the connection if pooling is enabled.
62.532 + ctx.db.commit()
62.533 + if unload and self.has_pooling:
62.534 + self._unload_context(self._ctx)
62.535 +
62.536 + def rollback():
62.537 + # do db rollback and release the connection if pooling is enabled.
62.538 + ctx.db.rollback()
62.539 + if self.has_pooling:
62.540 + self._unload_context(self._ctx)
62.541 +
62.542 + ctx.commit = commit
62.543 + ctx.rollback = rollback
62.544 +
62.545 + def _unload_context(self, ctx):
62.546 + del ctx.db
62.547 +
62.548 + def _connect(self, keywords):
62.549 + return self.db_module.connect(**keywords)
62.550 +
62.551 + def _connect_with_pooling(self, keywords):
62.552 + def get_pooled_db():
62.553 + from DBUtils import PooledDB
62.554 +
62.555 + # In DBUtils 0.9.3, `dbapi` argument is renamed as `creator`
62.556 + # see Bug#122112
62.557 +
62.558 + if PooledDB.__version__.split('.') < '0.9.3'.split('.'):
62.559 + return PooledDB.PooledDB(dbapi=self.db_module, **keywords)
62.560 + else:
62.561 + return PooledDB.PooledDB(creator=self.db_module, **keywords)
62.562 +
62.563 + if getattr(self, '_pooleddb', None) is None:
62.564 + self._pooleddb = get_pooled_db()
62.565 +
62.566 + return self._pooleddb.connection()
62.567 +
62.568 + def _db_cursor(self):
62.569 + return self.ctx.db.cursor()
62.570 +
62.571 + def _param_marker(self):
62.572 + """Returns parameter marker based on paramstyle attribute if this database."""
62.573 + style = getattr(self, 'paramstyle', 'pyformat')
62.574 +
62.575 + if style == 'qmark':
62.576 + return '?'
62.577 + elif style == 'numeric':
62.578 + return ':1'
62.579 + elif style in ['format', 'pyformat']:
62.580 + return '%s'
62.581 + raise UnknownParamstyle, style
62.582 +
62.583 + def _db_execute(self, cur, sql_query):
62.584 + """executes an sql query"""
62.585 + self.ctx.dbq_count += 1
62.586 +
62.587 + try:
62.588 + a = time.time()
62.589 + query, params = self._process_query(sql_query)
62.590 + out = cur.execute(query, params)
62.591 + b = time.time()
62.592 + except:
62.593 + if self.printing:
62.594 + print >> debug, 'ERR:', str(sql_query)
62.595 + if self.ctx.transactions:
62.596 + self.ctx.transactions[-1].rollback()
62.597 + else:
62.598 + self.ctx.rollback()
62.599 + raise
62.600 +
62.601 + if self.printing:
62.602 + print >> debug, '%s (%s): %s' % (round(b-a, 2), self.ctx.dbq_count, str(sql_query))
62.603 + return out
62.604 +
62.605 + def _process_query(self, sql_query):
62.606 + """Takes the SQLQuery object and returns query string and parameters.
62.607 + """
62.608 + paramstyle = getattr(self, 'paramstyle', 'pyformat')
62.609 + query = sql_query.query(paramstyle)
62.610 + params = sql_query.values()
62.611 + return query, params
62.612 +
62.613 + def _where(self, where, vars):
62.614 + if isinstance(where, (int, long)):
62.615 + where = "id = " + sqlparam(where)
62.616 + #@@@ for backward-compatibility
62.617 + elif isinstance(where, (list, tuple)) and len(where) == 2:
62.618 + where = SQLQuery(where[0], where[1])
62.619 + elif isinstance(where, SQLQuery):
62.620 + pass
62.621 + else:
62.622 + where = reparam(where, vars)
62.623 + return where
62.624 +
62.625 + def query(self, sql_query, vars=None, processed=False, _test=False):
62.626 + """
62.627 + Execute SQL query `sql_query` using dictionary `vars` to interpolate it.
62.628 + If `processed=True`, `vars` is a `reparam`-style list to use
62.629 + instead of interpolating.
62.630 +
62.631 + >>> db = DB(None, {})
62.632 + >>> db.query("SELECT * FROM foo", _test=True)
62.633 + <sql: 'SELECT * FROM foo'>
62.634 + >>> db.query("SELECT * FROM foo WHERE x = $x", vars=dict(x='f'), _test=True)
62.635 + <sql: "SELECT * FROM foo WHERE x = 'f'">
62.636 + >>> db.query("SELECT * FROM foo WHERE x = " + sqlquote('f'), _test=True)
62.637 + <sql: "SELECT * FROM foo WHERE x = 'f'">
62.638 + """
62.639 + if vars is None: vars = {}
62.640 +
62.641 + if not processed and not isinstance(sql_query, SQLQuery):
62.642 + sql_query = reparam(sql_query, vars)
62.643 +
62.644 + if _test: return sql_query
62.645 +
62.646 + db_cursor = self._db_cursor()
62.647 + self._db_execute(db_cursor, sql_query)
62.648 +
62.649 + if db_cursor.description:
62.650 + names = [x[0] for x in db_cursor.description]
62.651 + def iterwrapper():
62.652 + row = db_cursor.fetchone()
62.653 + while row:
62.654 + yield storage(dict(zip(names, row)))
62.655 + row = db_cursor.fetchone()
62.656 + out = iterbetter(iterwrapper())
62.657 + out.__len__ = lambda: int(db_cursor.rowcount)
62.658 + out.list = lambda: [storage(dict(zip(names, x))) \
62.659 + for x in db_cursor.fetchall()]
62.660 + else:
62.661 + out = db_cursor.rowcount
62.662 +
62.663 + if not self.ctx.transactions:
62.664 + self.ctx.commit()
62.665 + return out
62.666 +
62.667 + def select(self, tables, vars=None, what='*', where=None, order=None, group=None,
62.668 + limit=None, offset=None, _test=False):
62.669 + """
62.670 + Selects `what` from `tables` with clauses `where`, `order`,
62.671 + `group`, `limit`, and `offset`. Uses vars to interpolate.
62.672 + Otherwise, each clause can be a SQLQuery.
62.673 +
62.674 + >>> db = DB(None, {})
62.675 + >>> db.select('foo', _test=True)
62.676 + <sql: 'SELECT * FROM foo'>
62.677 + >>> db.select(['foo', 'bar'], where="foo.bar_id = bar.id", limit=5, _test=True)
62.678 + <sql: 'SELECT * FROM foo, bar WHERE foo.bar_id = bar.id LIMIT 5'>
62.679 + """
62.680 + if vars is None: vars = {}
62.681 + sql_clauses = self.sql_clauses(what, tables, where, group, order, limit, offset)
62.682 + clauses = [self.gen_clause(sql, val, vars) for sql, val in sql_clauses if val is not None]
62.683 + qout = SQLQuery.join(clauses)
62.684 + if _test: return qout
62.685 + return self.query(qout, processed=True)
62.686 +
62.687 + def where(self, table, what='*', order=None, group=None, limit=None,
62.688 + offset=None, _test=False, **kwargs):
62.689 + """
62.690 + Selects from `table` where keys are equal to values in `kwargs`.
62.691 +
62.692 + >>> db = DB(None, {})
62.693 + >>> db.where('foo', bar_id=3, _test=True)
62.694 + <sql: 'SELECT * FROM foo WHERE bar_id = 3'>
62.695 + >>> db.where('foo', source=2, crust='dewey', _test=True)
62.696 + <sql: "SELECT * FROM foo WHERE source = 2 AND crust = 'dewey'">
62.697 + >>> db.where('foo', _test=True)
62.698 + <sql: 'SELECT * FROM foo'>
62.699 + """
62.700 + where_clauses = []
62.701 + for k, v in kwargs.iteritems():
62.702 + where_clauses.append(k + ' = ' + sqlquote(v))
62.703 +
62.704 + if where_clauses:
62.705 + where = SQLQuery.join(where_clauses, " AND ")
62.706 + else:
62.707 + where = None
62.708 +
62.709 + return self.select(table, what=what, order=order,
62.710 + group=group, limit=limit, offset=offset, _test=_test,
62.711 + where=where)
62.712 +
62.713 + def sql_clauses(self, what, tables, where, group, order, limit, offset):
62.714 + return (
62.715 + ('SELECT', what),
62.716 + ('FROM', sqllist(tables)),
62.717 + ('WHERE', where),
62.718 + ('GROUP BY', group),
62.719 + ('ORDER BY', order),
62.720 + ('LIMIT', limit),
62.721 + ('OFFSET', offset))
62.722 +
62.723 + def gen_clause(self, sql, val, vars):
62.724 + if isinstance(val, (int, long)):
62.725 + if sql == 'WHERE':
62.726 + nout = 'id = ' + sqlquote(val)
62.727 + else:
62.728 + nout = SQLQuery(val)
62.729 + #@@@
62.730 + elif isinstance(val, (list, tuple)) and len(val) == 2:
62.731 + nout = SQLQuery(val[0], val[1]) # backwards-compatibility
62.732 + elif isinstance(val, SQLQuery):
62.733 + nout = val
62.734 + else:
62.735 + nout = reparam(val, vars)
62.736 +
62.737 + def xjoin(a, b):
62.738 + if a and b: return a + ' ' + b
62.739 + else: return a or b
62.740 +
62.741 + return xjoin(sql, nout)
62.742 +
62.743 + def insert(self, tablename, seqname=None, _test=False, **values):
62.744 + """
62.745 + Inserts `values` into `tablename`. Returns current sequence ID.
62.746 + Set `seqname` to the ID if it's not the default, or to `False`
62.747 + if there isn't one.
62.748 +
62.749 + >>> db = DB(None, {})
62.750 + >>> q = db.insert('foo', name='bob', age=2, created=SQLLiteral('NOW()'), _test=True)
62.751 + >>> q
62.752 + <sql: "INSERT INTO foo (age, name, created) VALUES (2, 'bob', NOW())">
62.753 + >>> q.query()
62.754 + 'INSERT INTO foo (age, name, created) VALUES (%s, %s, NOW())'
62.755 + >>> q.values()
62.756 + [2, 'bob']
62.757 + """
62.758 + def q(x): return "(" + x + ")"
62.759 +
62.760 + if values:
62.761 + _keys = SQLQuery.join(values.keys(), ', ')
62.762 + _values = SQLQuery.join([sqlparam(v) for v in values.values()], ', ')
62.763 + sql_query = "INSERT INTO %s " % tablename + q(_keys) + ' VALUES ' + q(_values)
62.764 + else:
62.765 + sql_query = SQLQuery(self._get_insert_default_values_query(tablename))
62.766 +
62.767 + if _test: return sql_query
62.768 +
62.769 + db_cursor = self._db_cursor()
62.770 + if seqname is not False:
62.771 + sql_query = self._process_insert_query(sql_query, tablename, seqname)
62.772 +
62.773 + if isinstance(sql_query, tuple):
62.774 + # for some databases, a separate query has to be made to find
62.775 + # the id of the inserted row.
62.776 + q1, q2 = sql_query
62.777 + self._db_execute(db_cursor, q1)
62.778 + self._db_execute(db_cursor, q2)
62.779 + else:
62.780 + self._db_execute(db_cursor, sql_query)
62.781 +
62.782 + try:
62.783 + out = db_cursor.fetchone()[0]
62.784 + except Exception:
62.785 + out = None
62.786 +
62.787 + if not self.ctx.transactions:
62.788 + self.ctx.commit()
62.789 + return out
62.790 +
62.791 + def _get_insert_default_values_query(self, table):
62.792 + return "INSERT INTO %s DEFAULT VALUES" % table
62.793 +
62.794 + def multiple_insert(self, tablename, values, seqname=None, _test=False):
62.795 + """
62.796 + Inserts multiple rows into `tablename`. The `values` must be a list of dictioanries,
62.797 + one for each row to be inserted, each with the same set of keys.
62.798 + Returns the list of ids of the inserted rows.
62.799 + Set `seqname` to the ID if it's not the default, or to `False`
62.800 + if there isn't one.
62.801 +
62.802 + >>> db = DB(None, {})
62.803 + >>> db.supports_multiple_insert = True
62.804 + >>> values = [{"name": "foo", "email": "foo@example.com"}, {"name": "bar", "email": "bar@example.com"}]
62.805 + >>> db.multiple_insert('person', values=values, _test=True)
62.806 + <sql: "INSERT INTO person (name, email) VALUES ('foo', 'foo@example.com'), ('bar', 'bar@example.com')">
62.807 + """
62.808 + if not values:
62.809 + return []
62.810 +
62.811 + if not self.supports_multiple_insert:
62.812 + out = [self.insert(tablename, seqname=seqname, _test=_test, **v) for v in values]
62.813 + if seqname is False:
62.814 + return None
62.815 + else:
62.816 + return out
62.817 +
62.818 + keys = values[0].keys()
62.819 + #@@ make sure all keys are valid
62.820 +
62.821 + # make sure all rows have same keys.
62.822 + for v in values:
62.823 + if v.keys() != keys:
62.824 + raise ValueError, 'Bad data'
62.825 +
62.826 + sql_query = SQLQuery('INSERT INTO %s (%s) VALUES ' % (tablename, ', '.join(keys)))
62.827 +
62.828 + for i, row in enumerate(values):
62.829 + if i != 0:
62.830 + sql_query.append(", ")
62.831 + SQLQuery.join([SQLParam(row[k]) for k in keys], sep=", ", target=sql_query, prefix="(", suffix=")")
62.832 +
62.833 + if _test: return sql_query
62.834 +
62.835 + db_cursor = self._db_cursor()
62.836 + if seqname is not False:
62.837 + sql_query = self._process_insert_query(sql_query, tablename, seqname)
62.838 +
62.839 + if isinstance(sql_query, tuple):
62.840 + # for some databases, a separate query has to be made to find
62.841 + # the id of the inserted row.
62.842 + q1, q2 = sql_query
62.843 + self._db_execute(db_cursor, q1)
62.844 + self._db_execute(db_cursor, q2)
62.845 + else:
62.846 + self._db_execute(db_cursor, sql_query)
62.847 +
62.848 + try:
62.849 + out = db_cursor.fetchone()[0]
62.850 + out = range(out-len(values)+1, out+1)
62.851 + except Exception:
62.852 + out = None
62.853 +
62.854 + if not self.ctx.transactions:
62.855 + self.ctx.commit()
62.856 + return out
62.857 +
62.858 +
62.859 + def update(self, tables, where, vars=None, _test=False, **values):
62.860 + """
62.861 + Update `tables` with clause `where` (interpolated using `vars`)
62.862 + and setting `values`.
62.863 +
62.864 + >>> db = DB(None, {})
62.865 + >>> name = 'Joseph'
62.866 + >>> q = db.update('foo', where='name = $name', name='bob', age=2,
62.867 + ... created=SQLLiteral('NOW()'), vars=locals(), _test=True)
62.868 + >>> q
62.869 + <sql: "UPDATE foo SET age = 2, name = 'bob', created = NOW() WHERE name = 'Joseph'">
62.870 + >>> q.query()
62.871 + 'UPDATE foo SET age = %s, name = %s, created = NOW() WHERE name = %s'
62.872 + >>> q.values()
62.873 + [2, 'bob', 'Joseph']
62.874 + """
62.875 + if vars is None: vars = {}
62.876 + where = self._where(where, vars)
62.877 +
62.878 + query = (
62.879 + "UPDATE " + sqllist(tables) +
62.880 + " SET " + sqlwhere(values, ', ') +
62.881 + " WHERE " + where)
62.882 +
62.883 + if _test: return query
62.884 +
62.885 + db_cursor = self._db_cursor()
62.886 + self._db_execute(db_cursor, query)
62.887 + if not self.ctx.transactions:
62.888 + self.ctx.commit()
62.889 + return db_cursor.rowcount
62.890 +
62.891 + def delete(self, table, where, using=None, vars=None, _test=False):
62.892 + """
62.893 + Deletes from `table` with clauses `where` and `using`.
62.894 +
62.895 + >>> db = DB(None, {})
62.896 + >>> name = 'Joe'
62.897 + >>> db.delete('foo', where='name = $name', vars=locals(), _test=True)
62.898 + <sql: "DELETE FROM foo WHERE name = 'Joe'">
62.899 + """
62.900 + if vars is None: vars = {}
62.901 + where = self._where(where, vars)
62.902 +
62.903 + q = 'DELETE FROM ' + table
62.904 + if using: q += ' USING ' + sqllist(using)
62.905 + if where: q += ' WHERE ' + where
62.906 +
62.907 + if _test: return q
62.908 +
62.909 + db_cursor = self._db_cursor()
62.910 + self._db_execute(db_cursor, q)
62.911 + if not self.ctx.transactions:
62.912 + self.ctx.commit()
62.913 + return db_cursor.rowcount
62.914 +
62.915 + def _process_insert_query(self, query, tablename, seqname):
62.916 + return query
62.917 +
62.918 + def transaction(self):
62.919 + """Start a transaction."""
62.920 + return Transaction(self.ctx)
62.921 +
62.922 +class PostgresDB(DB):
62.923 + """Postgres driver."""
62.924 + def __init__(self, **keywords):
62.925 + if 'pw' in keywords:
62.926 + keywords['password'] = keywords.pop('pw')
62.927 +
62.928 + db_module = import_driver(["psycopg2", "psycopg", "pgdb"], preferred=keywords.pop('driver', None))
62.929 + if db_module.__name__ == "psycopg2":
62.930 + import psycopg2.extensions
62.931 + psycopg2.extensions.register_type(psycopg2.extensions.UNICODE)
62.932 +
62.933 + # if db is not provided postgres driver will take it from PGDATABASE environment variable
62.934 + if 'db' in keywords:
62.935 + keywords['database'] = keywords.pop('db')
62.936 +
62.937 + self.dbname = "postgres"
62.938 + self.paramstyle = db_module.paramstyle
62.939 + DB.__init__(self, db_module, keywords)
62.940 + self.supports_multiple_insert = True
62.941 + self._sequences = None
62.942 +
62.943 + def _process_insert_query(self, query, tablename, seqname):
62.944 + if seqname is None:
62.945 + # when seqname is not provided guess the seqname and make sure it exists
62.946 + seqname = tablename + "_id_seq"
62.947 + if seqname not in self._get_all_sequences():
62.948 + seqname = None
62.949 +
62.950 + if seqname:
62.951 + query += "; SELECT currval('%s')" % seqname
62.952 +
62.953 + return query
62.954 +
62.955 + def _get_all_sequences(self):
62.956 + """Query postgres to find names of all sequences used in this database."""
62.957 + if self._sequences is None:
62.958 + q = "SELECT c.relname FROM pg_class c WHERE c.relkind = 'S'"
62.959 + self._sequences = set([c.relname for c in self.query(q)])
62.960 + return self._sequences
62.961 +
62.962 + def _connect(self, keywords):
62.963 + conn = DB._connect(self, keywords)
62.964 + try:
62.965 + conn.set_client_encoding('UTF8')
62.966 + except AttributeError:
62.967 + # fallback for pgdb driver
62.968 + conn.cursor().execute("set client_encoding to 'UTF-8'")
62.969 + return conn
62.970 +
62.971 + def _connect_with_pooling(self, keywords):
62.972 + conn = DB._connect_with_pooling(self, keywords)
62.973 + conn._con._con.set_client_encoding('UTF8')
62.974 + return conn
62.975 +
62.976 +class MySQLDB(DB):
62.977 + def __init__(self, **keywords):
62.978 + import MySQLdb as db
62.979 + if 'pw' in keywords:
62.980 + keywords['passwd'] = keywords['pw']
62.981 + del keywords['pw']
62.982 +
62.983 + if 'charset' not in keywords:
62.984 + keywords['charset'] = 'utf8'
62.985 + elif keywords['charset'] is None:
62.986 + del keywords['charset']
62.987 +
62.988 + self.paramstyle = db.paramstyle = 'pyformat' # it's both, like psycopg
62.989 + self.dbname = "mysql"
62.990 + DB.__init__(self, db, keywords)
62.991 + self.supports_multiple_insert = True
62.992 +
62.993 + def _process_insert_query(self, query, tablename, seqname):
62.994 + return query, SQLQuery('SELECT last_insert_id();')
62.995 +
62.996 + def _get_insert_default_values_query(self, table):
62.997 + return "INSERT INTO %s () VALUES()" % table
62.998 +
62.999 +def import_driver(drivers, preferred=None):
62.1000 + """Import the first available driver or preferred driver.
62.1001 + """
62.1002 + if preferred:
62.1003 + drivers = [preferred]
62.1004 +
62.1005 + for d in drivers:
62.1006 + try:
62.1007 + return __import__(d, None, None, ['x'])
62.1008 + except ImportError:
62.1009 + pass
62.1010 + raise ImportError("Unable to import " + " or ".join(drivers))
62.1011 +
62.1012 +class SqliteDB(DB):
62.1013 + def __init__(self, **keywords):
62.1014 + db = import_driver(["sqlite3", "pysqlite2.dbapi2", "sqlite"], preferred=keywords.pop('driver', None))
62.1015 +
62.1016 + if db.__name__ in ["sqlite3", "pysqlite2.dbapi2"]:
62.1017 + db.paramstyle = 'qmark'
62.1018 +
62.1019 + # sqlite driver doesn't create datatime objects for timestamp columns unless `detect_types` option is passed.
62.1020 + # It seems to be supported in sqlite3 and pysqlite2 drivers, not surte about sqlite.
62.1021 + keywords.setdefault('detect_types', db.PARSE_DECLTYPES)
62.1022 +
62.1023 + self.paramstyle = db.paramstyle
62.1024 + keywords['database'] = keywords.pop('db')
62.1025 + keywords['pooling'] = False # sqlite don't allows connections to be shared by threads
62.1026 + self.dbname = "sqlite"
62.1027 + DB.__init__(self, db, keywords)
62.1028 +
62.1029 + def _process_insert_query(self, query, tablename, seqname):
62.1030 + return query, SQLQuery('SELECT last_insert_rowid();')
62.1031 +
62.1032 + def query(self, *a, **kw):
62.1033 + out = DB.query(self, *a, **kw)
62.1034 + if isinstance(out, iterbetter):
62.1035 + del out.__len__
62.1036 + return out
62.1037 +
62.1038 +class FirebirdDB(DB):
62.1039 + """Firebird Database.
62.1040 + """
62.1041 + def __init__(self, **keywords):
62.1042 + try:
62.1043 + import kinterbasdb as db
62.1044 + except Exception:
62.1045 + db = None
62.1046 + pass
62.1047 + if 'pw' in keywords:
62.1048 + keywords['passwd'] = keywords['pw']
62.1049 + del keywords['pw']
62.1050 + keywords['database'] = keywords['db']
62.1051 + del keywords['db']
62.1052 + DB.__init__(self, db, keywords)
62.1053 +
62.1054 + def delete(self, table, where=None, using=None, vars=None, _test=False):
62.1055 + # firebird doesn't support using clause
62.1056 + using=None
62.1057 + return DB.delete(self, table, where, using, vars, _test)
62.1058 +
62.1059 + def sql_clauses(self, what, tables, where, group, order, limit, offset):
62.1060 + return (
62.1061 + ('SELECT', ''),
62.1062 + ('FIRST', limit),
62.1063 + ('SKIP', offset),
62.1064 + ('', what),
62.1065 + ('FROM', sqllist(tables)),
62.1066 + ('WHERE', where),
62.1067 + ('GROUP BY', group),
62.1068 + ('ORDER BY', order)
62.1069 + )
62.1070 +
62.1071 +class MSSQLDB(DB):
62.1072 + def __init__(self, **keywords):
62.1073 + import pymssql as db
62.1074 + if 'pw' in keywords:
62.1075 + keywords['password'] = keywords.pop('pw')
62.1076 + keywords['database'] = keywords.pop('db')
62.1077 + self.dbname = "mssql"
62.1078 + DB.__init__(self, db, keywords)
62.1079 +
62.1080 + def _process_query(self, sql_query):
62.1081 + """Takes the SQLQuery object and returns query string and parameters.
62.1082 + """
62.1083 + # MSSQLDB expects params to be a tuple.
62.1084 + # Overwriting the default implementation to convert params to tuple.
62.1085 + paramstyle = getattr(self, 'paramstyle', 'pyformat')
62.1086 + query = sql_query.query(paramstyle)
62.1087 + params = sql_query.values()
62.1088 + return query, tuple(params)
62.1089 +
62.1090 + def sql_clauses(self, what, tables, where, group, order, limit, offset):
62.1091 + return (
62.1092 + ('SELECT', what),
62.1093 + ('TOP', limit),
62.1094 + ('FROM', sqllist(tables)),
62.1095 + ('WHERE', where),
62.1096 + ('GROUP BY', group),
62.1097 + ('ORDER BY', order),
62.1098 + ('OFFSET', offset))
62.1099 +
62.1100 + def _test(self):
62.1101 + """Test LIMIT.
62.1102 +
62.1103 + Fake presence of pymssql module for running tests.
62.1104 + >>> import sys
62.1105 + >>> sys.modules['pymssql'] = sys.modules['sys']
62.1106 +
62.1107 + MSSQL has TOP clause instead of LIMIT clause.
62.1108 + >>> db = MSSQLDB(db='test', user='joe', pw='secret')
62.1109 + >>> db.select('foo', limit=4, _test=True)
62.1110 + <sql: 'SELECT * TOP 4 FROM foo'>
62.1111 + """
62.1112 + pass
62.1113 +
62.1114 +class OracleDB(DB):
62.1115 + def __init__(self, **keywords):
62.1116 + import cx_Oracle as db
62.1117 + if 'pw' in keywords:
62.1118 + keywords['password'] = keywords.pop('pw')
62.1119 +
62.1120 + #@@ TODO: use db.makedsn if host, port is specified
62.1121 + keywords['dsn'] = keywords.pop('db')
62.1122 + self.dbname = 'oracle'
62.1123 + db.paramstyle = 'numeric'
62.1124 + self.paramstyle = db.paramstyle
62.1125 +
62.1126 + # oracle doesn't support pooling
62.1127 + keywords.pop('pooling', None)
62.1128 + DB.__init__(self, db, keywords)
62.1129 +
62.1130 + def _process_insert_query(self, query, tablename, seqname):
62.1131 + if seqname is None:
62.1132 + # It is not possible to get seq name from table name in Oracle
62.1133 + return query
62.1134 + else:
62.1135 + return query + "; SELECT %s.currval FROM dual" % seqname
62.1136 +
62.1137 +_databases = {}
62.1138 +def database(dburl=None, **params):
62.1139 + """Creates appropriate database using params.
62.1140 +
62.1141 + Pooling will be enabled if DBUtils module is available.
62.1142 + Pooling can be disabled by passing pooling=False in params.
62.1143 + """
62.1144 + dbn = params.pop('dbn')
62.1145 + if dbn in _databases:
62.1146 + return _databases[dbn](**params)
62.1147 + else:
62.1148 + raise UnknownDB, dbn
62.1149 +
62.1150 +def register_database(name, clazz):
62.1151 + """
62.1152 + Register a database.
62.1153 +
62.1154 + >>> class LegacyDB(DB):
62.1155 + ... def __init__(self, **params):
62.1156 + ... pass
62.1157 + ...
62.1158 + >>> register_database('legacy', LegacyDB)
62.1159 + >>> db = database(dbn='legacy', db='test', user='joe', passwd='secret')
62.1160 + """
62.1161 + _databases[name] = clazz
62.1162 +
62.1163 +register_database('mysql', MySQLDB)
62.1164 +register_database('postgres', PostgresDB)
62.1165 +register_database('sqlite', SqliteDB)
62.1166 +register_database('firebird', FirebirdDB)
62.1167 +register_database('mssql', MSSQLDB)
62.1168 +register_database('oracle', OracleDB)
62.1169 +
62.1170 +def _interpolate(format):
62.1171 + """
62.1172 + Takes a format string and returns a list of 2-tuples of the form
62.1173 + (boolean, string) where boolean says whether string should be evaled
62.1174 + or not.
62.1175 +
62.1176 + from <http://lfw.org/python/Itpl.py> (public domain, Ka-Ping Yee)
62.1177 + """
62.1178 + from tokenize import tokenprog
62.1179 +
62.1180 + def matchorfail(text, pos):
62.1181 + match = tokenprog.match(text, pos)
62.1182 + if match is None:
62.1183 + raise _ItplError(text, pos)
62.1184 + return match, match.end()
62.1185 +
62.1186 + namechars = "abcdefghijklmnopqrstuvwxyz" \
62.1187 + "ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789_";
62.1188 + chunks = []
62.1189 + pos = 0
62.1190 +
62.1191 + while 1:
62.1192 + dollar = format.find("$", pos)
62.1193 + if dollar < 0:
62.1194 + break
62.1195 + nextchar = format[dollar + 1]
62.1196 +
62.1197 + if nextchar == "{":
62.1198 + chunks.append((0, format[pos:dollar]))
62.1199 + pos, level = dollar + 2, 1
62.1200 + while level:
62.1201 + match, pos = matchorfail(format, pos)
62.1202 + tstart, tend = match.regs[3]
62.1203 + token = format[tstart:tend]
62.1204 + if token == "{":
62.1205 + level = level + 1
62.1206 + elif token == "}":
62.1207 + level = level - 1
62.1208 + chunks.append((1, format[dollar + 2:pos - 1]))
62.1209 +
62.1210 + elif nextchar in namechars:
62.1211 + chunks.append((0, format[pos:dollar]))
62.1212 + match, pos = matchorfail(format, dollar + 1)
62.1213 + while pos < len(format):
62.1214 + if format[pos] == "." and \
62.1215 + pos + 1 < len(format) and format[pos + 1] in namechars:
62.1216 + match, pos = matchorfail(format, pos + 1)
62.1217 + elif format[pos] in "([":
62.1218 + pos, level = pos + 1, 1
62.1219 + while level:
62.1220 + match, pos = matchorfail(format, pos)
62.1221 + tstart, tend = match.regs[3]
62.1222 + token = format[tstart:tend]
62.1223 + if token[0] in "([":
62.1224 + level = level + 1
62.1225 + elif token[0] in ")]":
62.1226 + level = level - 1
62.1227 + else:
62.1228 + break
62.1229 + chunks.append((1, format[dollar + 1:pos]))
62.1230 + else:
62.1231 + chunks.append((0, format[pos:dollar + 1]))
62.1232 + pos = dollar + 1 + (nextchar == "$")
62.1233 +
62.1234 + if pos < len(format):
62.1235 + chunks.append((0, format[pos:]))
62.1236 + return chunks
62.1237 +
62.1238 +if __name__ == "__main__":
62.1239 + import doctest
62.1240 + doctest.testmod()
63.1 Binary file OpenSecurity/install/web.py-0.37/web/db.pyc has changed
64.1 --- /dev/null Thu Jan 01 00:00:00 1970 +0000
64.2 +++ b/OpenSecurity/install/web.py-0.37/web/debugerror.py Mon Dec 02 14:02:05 2013 +0100
64.3 @@ -0,0 +1,354 @@
64.4 +"""
64.5 +pretty debug errors
64.6 +(part of web.py)
64.7 +
64.8 +portions adapted from Django <djangoproject.com>
64.9 +Copyright (c) 2005, the Lawrence Journal-World
64.10 +Used under the modified BSD license:
64.11 +http://www.xfree86.org/3.3.6/COPYRIGHT2.html#5
64.12 +"""
64.13 +
64.14 +__all__ = ["debugerror", "djangoerror", "emailerrors"]
64.15 +
64.16 +import sys, urlparse, pprint, traceback
64.17 +from template import Template
64.18 +from net import websafe
64.19 +from utils import sendmail, safestr
64.20 +import webapi as web
64.21 +
64.22 +import os, os.path
64.23 +whereami = os.path.join(os.getcwd(), __file__)
64.24 +whereami = os.path.sep.join(whereami.split(os.path.sep)[:-1])
64.25 +djangoerror_t = """\
64.26 +$def with (exception_type, exception_value, frames)
64.27 +<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
64.28 +<html lang="en">
64.29 +<head>
64.30 + <meta http-equiv="content-type" content="text/html; charset=utf-8" />
64.31 + <meta name="robots" content="NONE,NOARCHIVE" />
64.32 + <title>$exception_type at $ctx.path</title>
64.33 + <style type="text/css">
64.34 + html * { padding:0; margin:0; }
64.35 + body * { padding:10px 20px; }
64.36 + body * * { padding:0; }
64.37 + body { font:small sans-serif; }
64.38 + body>div { border-bottom:1px solid #ddd; }
64.39 + h1 { font-weight:normal; }
64.40 + h2 { margin-bottom:.8em; }
64.41 + h2 span { font-size:80%; color:#666; font-weight:normal; }
64.42 + h3 { margin:1em 0 .5em 0; }
64.43 + h4 { margin:0 0 .5em 0; font-weight: normal; }
64.44 + table {
64.45 + border:1px solid #ccc; border-collapse: collapse; background:white; }
64.46 + tbody td, tbody th { vertical-align:top; padding:2px 3px; }
64.47 + thead th {
64.48 + padding:1px 6px 1px 3px; background:#fefefe; text-align:left;
64.49 + font-weight:normal; font-size:11px; border:1px solid #ddd; }
64.50 + tbody th { text-align:right; color:#666; padding-right:.5em; }
64.51 + table.vars { margin:5px 0 2px 40px; }
64.52 + table.vars td, table.req td { font-family:monospace; }
64.53 + table td.code { width:100%;}
64.54 + table td.code div { overflow:hidden; }
64.55 + table.source th { color:#666; }
64.56 + table.source td {
64.57 + font-family:monospace; white-space:pre; border-bottom:1px solid #eee; }
64.58 + ul.traceback { list-style-type:none; }
64.59 + ul.traceback li.frame { margin-bottom:1em; }
64.60 + div.context { margin: 10px 0; }
64.61 + div.context ol {
64.62 + padding-left:30px; margin:0 10px; list-style-position: inside; }
64.63 + div.context ol li {
64.64 + font-family:monospace; white-space:pre; color:#666; cursor:pointer; }
64.65 + div.context ol.context-line li { color:black; background-color:#ccc; }
64.66 + div.context ol.context-line li span { float: right; }
64.67 + div.commands { margin-left: 40px; }
64.68 + div.commands a { color:black; text-decoration:none; }
64.69 + #summary { background: #ffc; }
64.70 + #summary h2 { font-weight: normal; color: #666; }
64.71 + #explanation { background:#eee; }
64.72 + #template, #template-not-exist { background:#f6f6f6; }
64.73 + #template-not-exist ul { margin: 0 0 0 20px; }
64.74 + #traceback { background:#eee; }
64.75 + #requestinfo { background:#f6f6f6; padding-left:120px; }
64.76 + #summary table { border:none; background:transparent; }
64.77 + #requestinfo h2, #requestinfo h3 { position:relative; margin-left:-100px; }
64.78 + #requestinfo h3 { margin-bottom:-1em; }
64.79 + .error { background: #ffc; }
64.80 + .specific { color:#cc3300; font-weight:bold; }
64.81 + </style>
64.82 + <script type="text/javascript">
64.83 + //<!--
64.84 + function getElementsByClassName(oElm, strTagName, strClassName){
64.85 + // Written by Jonathan Snook, http://www.snook.ca/jon;
64.86 + // Add-ons by Robert Nyman, http://www.robertnyman.com
64.87 + var arrElements = (strTagName == "*" && document.all)? document.all :
64.88 + oElm.getElementsByTagName(strTagName);
64.89 + var arrReturnElements = new Array();
64.90 + strClassName = strClassName.replace(/\-/g, "\\-");
64.91 + var oRegExp = new RegExp("(^|\\s)" + strClassName + "(\\s|$$)");
64.92 + var oElement;
64.93 + for(var i=0; i<arrElements.length; i++){
64.94 + oElement = arrElements[i];
64.95 + if(oRegExp.test(oElement.className)){
64.96 + arrReturnElements.push(oElement);
64.97 + }
64.98 + }
64.99 + return (arrReturnElements)
64.100 + }
64.101 + function hideAll(elems) {
64.102 + for (var e = 0; e < elems.length; e++) {
64.103 + elems[e].style.display = 'none';
64.104 + }
64.105 + }
64.106 + window.onload = function() {
64.107 + hideAll(getElementsByClassName(document, 'table', 'vars'));
64.108 + hideAll(getElementsByClassName(document, 'ol', 'pre-context'));
64.109 + hideAll(getElementsByClassName(document, 'ol', 'post-context'));
64.110 + }
64.111 + function toggle() {
64.112 + for (var i = 0; i < arguments.length; i++) {
64.113 + var e = document.getElementById(arguments[i]);
64.114 + if (e) {
64.115 + e.style.display = e.style.display == 'none' ? 'block' : 'none';
64.116 + }
64.117 + }
64.118 + return false;
64.119 + }
64.120 + function varToggle(link, id) {
64.121 + toggle('v' + id);
64.122 + var s = link.getElementsByTagName('span')[0];
64.123 + var uarr = String.fromCharCode(0x25b6);
64.124 + var darr = String.fromCharCode(0x25bc);
64.125 + s.innerHTML = s.innerHTML == uarr ? darr : uarr;
64.126 + return false;
64.127 + }
64.128 + //-->
64.129 + </script>
64.130 +</head>
64.131 +<body>
64.132 +
64.133 +$def dicttable (d, kls='req', id=None):
64.134 + $ items = d and d.items() or []
64.135 + $items.sort()
64.136 + $:dicttable_items(items, kls, id)
64.137 +
64.138 +$def dicttable_items(items, kls='req', id=None):
64.139 + $if items:
64.140 + <table class="$kls"
64.141 + $if id: id="$id"
64.142 + ><thead><tr><th>Variable</th><th>Value</th></tr></thead>
64.143 + <tbody>
64.144 + $for k, v in items:
64.145 + <tr><td>$k</td><td class="code"><div>$prettify(v)</div></td></tr>
64.146 + </tbody>
64.147 + </table>
64.148 + $else:
64.149 + <p>No data.</p>
64.150 +
64.151 +<div id="summary">
64.152 + <h1>$exception_type at $ctx.path</h1>
64.153 + <h2>$exception_value</h2>
64.154 + <table><tr>
64.155 + <th>Python</th>
64.156 + <td>$frames[0].filename in $frames[0].function, line $frames[0].lineno</td>
64.157 + </tr><tr>
64.158 + <th>Web</th>
64.159 + <td>$ctx.method $ctx.home$ctx.path</td>
64.160 + </tr></table>
64.161 +</div>
64.162 +<div id="traceback">
64.163 +<h2>Traceback <span>(innermost first)</span></h2>
64.164 +<ul class="traceback">
64.165 +$for frame in frames:
64.166 + <li class="frame">
64.167 + <code>$frame.filename</code> in <code>$frame.function</code>
64.168 + $if frame.context_line is not None:
64.169 + <div class="context" id="c$frame.id">
64.170 + $if frame.pre_context:
64.171 + <ol start="$frame.pre_context_lineno" class="pre-context" id="pre$frame.id">
64.172 + $for line in frame.pre_context:
64.173 + <li onclick="toggle('pre$frame.id', 'post$frame.id')">$line</li>
64.174 + </ol>
64.175 + <ol start="$frame.lineno" class="context-line"><li onclick="toggle('pre$frame.id', 'post$frame.id')">$frame.context_line <span>...</span></li></ol>
64.176 + $if frame.post_context:
64.177 + <ol start='${frame.lineno + 1}' class="post-context" id="post$frame.id">
64.178 + $for line in frame.post_context:
64.179 + <li onclick="toggle('pre$frame.id', 'post$frame.id')">$line</li>
64.180 + </ol>
64.181 + </div>
64.182 +
64.183 + $if frame.vars:
64.184 + <div class="commands">
64.185 + <a href='#' onclick="return varToggle(this, '$frame.id')"><span>▶</span> Local vars</a>
64.186 + $# $inspect.formatargvalues(*inspect.getargvalues(frame['tb'].tb_frame))
64.187 + </div>
64.188 + $:dicttable(frame.vars, kls='vars', id=('v' + str(frame.id)))
64.189 + </li>
64.190 + </ul>
64.191 +</div>
64.192 +
64.193 +<div id="requestinfo">
64.194 +$if ctx.output or ctx.headers:
64.195 + <h2>Response so far</h2>
64.196 + <h3>HEADERS</h3>
64.197 + $:dicttable_items(ctx.headers)
64.198 +
64.199 + <h3>BODY</h3>
64.200 + <p class="req" style="padding-bottom: 2em"><code>
64.201 + $ctx.output
64.202 + </code></p>
64.203 +
64.204 +<h2>Request information</h2>
64.205 +
64.206 +<h3>INPUT</h3>
64.207 +$:dicttable(web.input(_unicode=False))
64.208 +
64.209 +<h3 id="cookie-info">COOKIES</h3>
64.210 +$:dicttable(web.cookies())
64.211 +
64.212 +<h3 id="meta-info">META</h3>
64.213 +$ newctx = [(k, v) for (k, v) in ctx.iteritems() if not k.startswith('_') and not isinstance(v, dict)]
64.214 +$:dicttable(dict(newctx))
64.215 +
64.216 +<h3 id="meta-info">ENVIRONMENT</h3>
64.217 +$:dicttable(ctx.env)
64.218 +</div>
64.219 +
64.220 +<div id="explanation">
64.221 + <p>
64.222 + You're seeing this error because you have <code>web.config.debug</code>
64.223 + set to <code>True</code>. Set that to <code>False</code> if you don't want to see this.
64.224 + </p>
64.225 +</div>
64.226 +
64.227 +</body>
64.228 +</html>
64.229 +"""
64.230 +
64.231 +djangoerror_r = None
64.232 +
64.233 +def djangoerror():
64.234 + def _get_lines_from_file(filename, lineno, context_lines):
64.235 + """
64.236 + Returns context_lines before and after lineno from file.
64.237 + Returns (pre_context_lineno, pre_context, context_line, post_context).
64.238 + """
64.239 + try:
64.240 + source = open(filename).readlines()
64.241 + lower_bound = max(0, lineno - context_lines)
64.242 + upper_bound = lineno + context_lines
64.243 +
64.244 + pre_context = \
64.245 + [line.strip('\n') for line in source[lower_bound:lineno]]
64.246 + context_line = source[lineno].strip('\n')
64.247 + post_context = \
64.248 + [line.strip('\n') for line in source[lineno + 1:upper_bound]]
64.249 +
64.250 + return lower_bound, pre_context, context_line, post_context
64.251 + except (OSError, IOError, IndexError):
64.252 + return None, [], None, []
64.253 +
64.254 + exception_type, exception_value, tback = sys.exc_info()
64.255 + frames = []
64.256 + while tback is not None:
64.257 + filename = tback.tb_frame.f_code.co_filename
64.258 + function = tback.tb_frame.f_code.co_name
64.259 + lineno = tback.tb_lineno - 1
64.260 +
64.261 + # hack to get correct line number for templates
64.262 + lineno += tback.tb_frame.f_locals.get("__lineoffset__", 0)
64.263 +
64.264 + pre_context_lineno, pre_context, context_line, post_context = \
64.265 + _get_lines_from_file(filename, lineno, 7)
64.266 +
64.267 + if '__hidetraceback__' not in tback.tb_frame.f_locals:
64.268 + frames.append(web.storage({
64.269 + 'tback': tback,
64.270 + 'filename': filename,
64.271 + 'function': function,
64.272 + 'lineno': lineno,
64.273 + 'vars': tback.tb_frame.f_locals,
64.274 + 'id': id(tback),
64.275 + 'pre_context': pre_context,
64.276 + 'context_line': context_line,
64.277 + 'post_context': post_context,
64.278 + 'pre_context_lineno': pre_context_lineno,
64.279 + }))
64.280 + tback = tback.tb_next
64.281 + frames.reverse()
64.282 + urljoin = urlparse.urljoin
64.283 + def prettify(x):
64.284 + try:
64.285 + out = pprint.pformat(x)
64.286 + except Exception, e:
64.287 + out = '[could not display: <' + e.__class__.__name__ + \
64.288 + ': '+str(e)+'>]'
64.289 + return out
64.290 +
64.291 + global djangoerror_r
64.292 + if djangoerror_r is None:
64.293 + djangoerror_r = Template(djangoerror_t, filename=__file__, filter=websafe)
64.294 +
64.295 + t = djangoerror_r
64.296 + globals = {'ctx': web.ctx, 'web':web, 'dict':dict, 'str':str, 'prettify': prettify}
64.297 + t.t.func_globals.update(globals)
64.298 + return t(exception_type, exception_value, frames)
64.299 +
64.300 +def debugerror():
64.301 + """
64.302 + A replacement for `internalerror` that presents a nice page with lots
64.303 + of debug information for the programmer.
64.304 +
64.305 + (Based on the beautiful 500 page from [Django](http://djangoproject.com/),
64.306 + designed by [Wilson Miner](http://wilsonminer.com/).)
64.307 + """
64.308 + return web._InternalError(djangoerror())
64.309 +
64.310 +def emailerrors(to_address, olderror, from_address=None):
64.311 + """
64.312 + Wraps the old `internalerror` handler (pass as `olderror`) to
64.313 + additionally email all errors to `to_address`, to aid in
64.314 + debugging production websites.
64.315 +
64.316 + Emails contain a normal text traceback as well as an
64.317 + attachment containing the nice `debugerror` page.
64.318 + """
64.319 + from_address = from_address or to_address
64.320 +
64.321 + def emailerrors_internal():
64.322 + error = olderror()
64.323 + tb = sys.exc_info()
64.324 + error_name = tb[0]
64.325 + error_value = tb[1]
64.326 + tb_txt = ''.join(traceback.format_exception(*tb))
64.327 + path = web.ctx.path
64.328 + request = web.ctx.method + ' ' + web.ctx.home + web.ctx.fullpath
64.329 +
64.330 + message = "\n%s\n\n%s\n\n" % (request, tb_txt)
64.331 +
64.332 + sendmail(
64.333 + "your buggy site <%s>" % from_address,
64.334 + "the bugfixer <%s>" % to_address,
64.335 + "bug: %(error_name)s: %(error_value)s (%(path)s)" % locals(),
64.336 + message,
64.337 + attachments=[
64.338 + dict(filename="bug.html", content=safestr(djangoerror()))
64.339 + ],
64.340 + )
64.341 + return error
64.342 +
64.343 + return emailerrors_internal
64.344 +
64.345 +if __name__ == "__main__":
64.346 + urls = (
64.347 + '/', 'index'
64.348 + )
64.349 + from application import application
64.350 + app = application(urls, globals())
64.351 + app.internalerror = debugerror
64.352 +
64.353 + class index:
64.354 + def GET(self):
64.355 + thisdoesnotexist
64.356 +
64.357 + app.run()
65.1 Binary file OpenSecurity/install/web.py-0.37/web/debugerror.pyc has changed
66.1 --- /dev/null Thu Jan 01 00:00:00 1970 +0000
66.2 +++ b/OpenSecurity/install/web.py-0.37/web/form.py Mon Dec 02 14:02:05 2013 +0100
66.3 @@ -0,0 +1,410 @@
66.4 +"""
66.5 +HTML forms
66.6 +(part of web.py)
66.7 +"""
66.8 +
66.9 +import copy, re
66.10 +import webapi as web
66.11 +import utils, net
66.12 +
66.13 +def attrget(obj, attr, value=None):
66.14 + try:
66.15 + if hasattr(obj, 'has_key') and obj.has_key(attr):
66.16 + return obj[attr]
66.17 + except TypeError:
66.18 + # Handle the case where has_key takes different number of arguments.
66.19 + # This is the case with Model objects on appengine. See #134
66.20 + pass
66.21 + if hasattr(obj, attr):
66.22 + return getattr(obj, attr)
66.23 + return value
66.24 +
66.25 +class Form(object):
66.26 + r"""
66.27 + HTML form.
66.28 +
66.29 + >>> f = Form(Textbox("x"))
66.30 + >>> f.render()
66.31 + u'<table>\n <tr><th><label for="x">x</label></th><td><input type="text" id="x" name="x"/></td></tr>\n</table>'
66.32 + """
66.33 + def __init__(self, *inputs, **kw):
66.34 + self.inputs = inputs
66.35 + self.valid = True
66.36 + self.note = None
66.37 + self.validators = kw.pop('validators', [])
66.38 +
66.39 + def __call__(self, x=None):
66.40 + o = copy.deepcopy(self)
66.41 + if x: o.validates(x)
66.42 + return o
66.43 +
66.44 + def render(self):
66.45 + out = ''
66.46 + out += self.rendernote(self.note)
66.47 + out += '<table>\n'
66.48 +
66.49 + for i in self.inputs:
66.50 + html = utils.safeunicode(i.pre) + i.render() + self.rendernote(i.note) + utils.safeunicode(i.post)
66.51 + if i.is_hidden():
66.52 + out += ' <tr style="display: none;"><th></th><td>%s</td></tr>\n' % (html)
66.53 + else:
66.54 + out += ' <tr><th><label for="%s">%s</label></th><td>%s</td></tr>\n' % (i.id, net.websafe(i.description), html)
66.55 + out += "</table>"
66.56 + return out
66.57 +
66.58 + def render_css(self):
66.59 + out = []
66.60 + out.append(self.rendernote(self.note))
66.61 + for i in self.inputs:
66.62 + if not i.is_hidden():
66.63 + out.append('<label for="%s">%s</label>' % (i.id, net.websafe(i.description)))
66.64 + out.append(i.pre)
66.65 + out.append(i.render())
66.66 + out.append(self.rendernote(i.note))
66.67 + out.append(i.post)
66.68 + out.append('\n')
66.69 + return ''.join(out)
66.70 +
66.71 + def rendernote(self, note):
66.72 + if note: return '<strong class="wrong">%s</strong>' % net.websafe(note)
66.73 + else: return ""
66.74 +
66.75 + def validates(self, source=None, _validate=True, **kw):
66.76 + source = source or kw or web.input()
66.77 + out = True
66.78 + for i in self.inputs:
66.79 + v = attrget(source, i.name)
66.80 + if _validate:
66.81 + out = i.validate(v) and out
66.82 + else:
66.83 + i.set_value(v)
66.84 + if _validate:
66.85 + out = out and self._validate(source)
66.86 + self.valid = out
66.87 + return out
66.88 +
66.89 + def _validate(self, value):
66.90 + self.value = value
66.91 + for v in self.validators:
66.92 + if not v.valid(value):
66.93 + self.note = v.msg
66.94 + return False
66.95 + return True
66.96 +
66.97 + def fill(self, source=None, **kw):
66.98 + return self.validates(source, _validate=False, **kw)
66.99 +
66.100 + def __getitem__(self, i):
66.101 + for x in self.inputs:
66.102 + if x.name == i: return x
66.103 + raise KeyError, i
66.104 +
66.105 + def __getattr__(self, name):
66.106 + # don't interfere with deepcopy
66.107 + inputs = self.__dict__.get('inputs') or []
66.108 + for x in inputs:
66.109 + if x.name == name: return x
66.110 + raise AttributeError, name
66.111 +
66.112 + def get(self, i, default=None):
66.113 + try:
66.114 + return self[i]
66.115 + except KeyError:
66.116 + return default
66.117 +
66.118 + def _get_d(self): #@@ should really be form.attr, no?
66.119 + return utils.storage([(i.name, i.get_value()) for i in self.inputs])
66.120 + d = property(_get_d)
66.121 +
66.122 +class Input(object):
66.123 + def __init__(self, name, *validators, **attrs):
66.124 + self.name = name
66.125 + self.validators = validators
66.126 + self.attrs = attrs = AttributeList(attrs)
66.127 +
66.128 + self.description = attrs.pop('description', name)
66.129 + self.value = attrs.pop('value', None)
66.130 + self.pre = attrs.pop('pre', "")
66.131 + self.post = attrs.pop('post', "")
66.132 + self.note = None
66.133 +
66.134 + self.id = attrs.setdefault('id', self.get_default_id())
66.135 +
66.136 + if 'class_' in attrs:
66.137 + attrs['class'] = attrs['class_']
66.138 + del attrs['class_']
66.139 +
66.140 + def is_hidden(self):
66.141 + return False
66.142 +
66.143 + def get_type(self):
66.144 + raise NotImplementedError
66.145 +
66.146 + def get_default_id(self):
66.147 + return self.name
66.148 +
66.149 + def validate(self, value):
66.150 + self.set_value(value)
66.151 +
66.152 + for v in self.validators:
66.153 + if not v.valid(value):
66.154 + self.note = v.msg
66.155 + return False
66.156 + return True
66.157 +
66.158 + def set_value(self, value):
66.159 + self.value = value
66.160 +
66.161 + def get_value(self):
66.162 + return self.value
66.163 +
66.164 + def render(self):
66.165 + attrs = self.attrs.copy()
66.166 + attrs['type'] = self.get_type()
66.167 + if self.value is not None:
66.168 + attrs['value'] = self.value
66.169 + attrs['name'] = self.name
66.170 + return '<input %s/>' % attrs
66.171 +
66.172 + def rendernote(self, note):
66.173 + if note: return '<strong class="wrong">%s</strong>' % net.websafe(note)
66.174 + else: return ""
66.175 +
66.176 + def addatts(self):
66.177 + # add leading space for backward-compatibility
66.178 + return " " + str(self.attrs)
66.179 +
66.180 +class AttributeList(dict):
66.181 + """List of atributes of input.
66.182 +
66.183 + >>> a = AttributeList(type='text', name='x', value=20)
66.184 + >>> a
66.185 + <attrs: 'type="text" name="x" value="20"'>
66.186 + """
66.187 + def copy(self):
66.188 + return AttributeList(self)
66.189 +
66.190 + def __str__(self):
66.191 + return " ".join(['%s="%s"' % (k, net.websafe(v)) for k, v in self.items()])
66.192 +
66.193 + def __repr__(self):
66.194 + return '<attrs: %s>' % repr(str(self))
66.195 +
66.196 +class Textbox(Input):
66.197 + """Textbox input.
66.198 +
66.199 + >>> Textbox(name='foo', value='bar').render()
66.200 + u'<input type="text" id="foo" value="bar" name="foo"/>'
66.201 + >>> Textbox(name='foo', value=0).render()
66.202 + u'<input type="text" id="foo" value="0" name="foo"/>'
66.203 + """
66.204 + def get_type(self):
66.205 + return 'text'
66.206 +
66.207 +class Password(Input):
66.208 + """Password input.
66.209 +
66.210 + >>> Password(name='password', value='secret').render()
66.211 + u'<input type="password" id="password" value="secret" name="password"/>'
66.212 + """
66.213 +
66.214 + def get_type(self):
66.215 + return 'password'
66.216 +
66.217 +class Textarea(Input):
66.218 + """Textarea input.
66.219 +
66.220 + >>> Textarea(name='foo', value='bar').render()
66.221 + u'<textarea id="foo" name="foo">bar</textarea>'
66.222 + """
66.223 + def render(self):
66.224 + attrs = self.attrs.copy()
66.225 + attrs['name'] = self.name
66.226 + value = net.websafe(self.value or '')
66.227 + return '<textarea %s>%s</textarea>' % (attrs, value)
66.228 +
66.229 +class Dropdown(Input):
66.230 + r"""Dropdown/select input.
66.231 +
66.232 + >>> Dropdown(name='foo', args=['a', 'b', 'c'], value='b').render()
66.233 + u'<select id="foo" name="foo">\n <option value="a">a</option>\n <option selected="selected" value="b">b</option>\n <option value="c">c</option>\n</select>\n'
66.234 + >>> Dropdown(name='foo', args=[('a', 'aa'), ('b', 'bb'), ('c', 'cc')], value='b').render()
66.235 + u'<select id="foo" name="foo">\n <option value="a">aa</option>\n <option selected="selected" value="b">bb</option>\n <option value="c">cc</option>\n</select>\n'
66.236 + """
66.237 + def __init__(self, name, args, *validators, **attrs):
66.238 + self.args = args
66.239 + super(Dropdown, self).__init__(name, *validators, **attrs)
66.240 +
66.241 + def render(self):
66.242 + attrs = self.attrs.copy()
66.243 + attrs['name'] = self.name
66.244 +
66.245 + x = '<select %s>\n' % attrs
66.246 +
66.247 + for arg in self.args:
66.248 + x += self._render_option(arg)
66.249 +
66.250 + x += '</select>\n'
66.251 + return x
66.252 +
66.253 + def _render_option(self, arg, indent=' '):
66.254 + if isinstance(arg, (tuple, list)):
66.255 + value, desc= arg
66.256 + else:
66.257 + value, desc = arg, arg
66.258 +
66.259 + if self.value == value or (isinstance(self.value, list) and value in self.value):
66.260 + select_p = ' selected="selected"'
66.261 + else:
66.262 + select_p = ''
66.263 + return indent + '<option%s value="%s">%s</option>\n' % (select_p, net.websafe(value), net.websafe(desc))
66.264 +
66.265 +
66.266 +class GroupedDropdown(Dropdown):
66.267 + r"""Grouped Dropdown/select input.
66.268 +
66.269 + >>> GroupedDropdown(name='car_type', args=(('Swedish Cars', ('Volvo', 'Saab')), ('German Cars', ('Mercedes', 'Audi'))), value='Audi').render()
66.270 + u'<select id="car_type" name="car_type">\n <optgroup label="Swedish Cars">\n <option value="Volvo">Volvo</option>\n <option value="Saab">Saab</option>\n </optgroup>\n <optgroup label="German Cars">\n <option value="Mercedes">Mercedes</option>\n <option selected="selected" value="Audi">Audi</option>\n </optgroup>\n</select>\n'
66.271 + >>> GroupedDropdown(name='car_type', args=(('Swedish Cars', (('v', 'Volvo'), ('s', 'Saab'))), ('German Cars', (('m', 'Mercedes'), ('a', 'Audi')))), value='a').render()
66.272 + u'<select id="car_type" name="car_type">\n <optgroup label="Swedish Cars">\n <option value="v">Volvo</option>\n <option value="s">Saab</option>\n </optgroup>\n <optgroup label="German Cars">\n <option value="m">Mercedes</option>\n <option selected="selected" value="a">Audi</option>\n </optgroup>\n</select>\n'
66.273 +
66.274 + """
66.275 + def __init__(self, name, args, *validators, **attrs):
66.276 + self.args = args
66.277 + super(Dropdown, self).__init__(name, *validators, **attrs)
66.278 +
66.279 + def render(self):
66.280 + attrs = self.attrs.copy()
66.281 + attrs['name'] = self.name
66.282 +
66.283 + x = '<select %s>\n' % attrs
66.284 +
66.285 + for label, options in self.args:
66.286 + x += ' <optgroup label="%s">\n' % net.websafe(label)
66.287 + for arg in options:
66.288 + x += self._render_option(arg, indent = ' ')
66.289 + x += ' </optgroup>\n'
66.290 +
66.291 + x += '</select>\n'
66.292 + return x
66.293 +
66.294 +class Radio(Input):
66.295 + def __init__(self, name, args, *validators, **attrs):
66.296 + self.args = args
66.297 + super(Radio, self).__init__(name, *validators, **attrs)
66.298 +
66.299 + def render(self):
66.300 + x = '<span>'
66.301 + for arg in self.args:
66.302 + if isinstance(arg, (tuple, list)):
66.303 + value, desc= arg
66.304 + else:
66.305 + value, desc = arg, arg
66.306 + attrs = self.attrs.copy()
66.307 + attrs['name'] = self.name
66.308 + attrs['type'] = 'radio'
66.309 + attrs['value'] = value
66.310 + if self.value == value:
66.311 + attrs['checked'] = 'checked'
66.312 + x += '<input %s/> %s' % (attrs, net.websafe(desc))
66.313 + x += '</span>'
66.314 + return x
66.315 +
66.316 +class Checkbox(Input):
66.317 + """Checkbox input.
66.318 +
66.319 + >>> Checkbox('foo', value='bar', checked=True).render()
66.320 + u'<input checked="checked" type="checkbox" id="foo_bar" value="bar" name="foo"/>'
66.321 + >>> Checkbox('foo', value='bar').render()
66.322 + u'<input type="checkbox" id="foo_bar" value="bar" name="foo"/>'
66.323 + >>> c = Checkbox('foo', value='bar')
66.324 + >>> c.validate('on')
66.325 + True
66.326 + >>> c.render()
66.327 + u'<input checked="checked" type="checkbox" id="foo_bar" value="bar" name="foo"/>'
66.328 + """
66.329 + def __init__(self, name, *validators, **attrs):
66.330 + self.checked = attrs.pop('checked', False)
66.331 + Input.__init__(self, name, *validators, **attrs)
66.332 +
66.333 + def get_default_id(self):
66.334 + value = utils.safestr(self.value or "")
66.335 + return self.name + '_' + value.replace(' ', '_')
66.336 +
66.337 + def render(self):
66.338 + attrs = self.attrs.copy()
66.339 + attrs['type'] = 'checkbox'
66.340 + attrs['name'] = self.name
66.341 + attrs['value'] = self.value
66.342 +
66.343 + if self.checked:
66.344 + attrs['checked'] = 'checked'
66.345 + return '<input %s/>' % attrs
66.346 +
66.347 + def set_value(self, value):
66.348 + self.checked = bool(value)
66.349 +
66.350 + def get_value(self):
66.351 + return self.checked
66.352 +
66.353 +class Button(Input):
66.354 + """HTML Button.
66.355 +
66.356 + >>> Button("save").render()
66.357 + u'<button id="save" name="save">save</button>'
66.358 + >>> Button("action", value="save", html="<b>Save Changes</b>").render()
66.359 + u'<button id="action" value="save" name="action"><b>Save Changes</b></button>'
66.360 + """
66.361 + def __init__(self, name, *validators, **attrs):
66.362 + super(Button, self).__init__(name, *validators, **attrs)
66.363 + self.description = ""
66.364 +
66.365 + def render(self):
66.366 + attrs = self.attrs.copy()
66.367 + attrs['name'] = self.name
66.368 + if self.value is not None:
66.369 + attrs['value'] = self.value
66.370 + html = attrs.pop('html', None) or net.websafe(self.name)
66.371 + return '<button %s>%s</button>' % (attrs, html)
66.372 +
66.373 +class Hidden(Input):
66.374 + """Hidden Input.
66.375 +
66.376 + >>> Hidden(name='foo', value='bar').render()
66.377 + u'<input type="hidden" id="foo" value="bar" name="foo"/>'
66.378 + """
66.379 + def is_hidden(self):
66.380 + return True
66.381 +
66.382 + def get_type(self):
66.383 + return 'hidden'
66.384 +
66.385 +class File(Input):
66.386 + """File input.
66.387 +
66.388 + >>> File(name='f').render()
66.389 + u'<input type="file" id="f" name="f"/>'
66.390 + """
66.391 + def get_type(self):
66.392 + return 'file'
66.393 +
66.394 +class Validator:
66.395 + def __deepcopy__(self, memo): return copy.copy(self)
66.396 + def __init__(self, msg, test, jstest=None): utils.autoassign(self, locals())
66.397 + def valid(self, value):
66.398 + try: return self.test(value)
66.399 + except: return False
66.400 +
66.401 +notnull = Validator("Required", bool)
66.402 +
66.403 +class regexp(Validator):
66.404 + def __init__(self, rexp, msg):
66.405 + self.rexp = re.compile(rexp)
66.406 + self.msg = msg
66.407 +
66.408 + def valid(self, value):
66.409 + return bool(self.rexp.match(value))
66.410 +
66.411 +if __name__ == "__main__":
66.412 + import doctest
66.413 + doctest.testmod()
67.1 Binary file OpenSecurity/install/web.py-0.37/web/form.pyc has changed
68.1 --- /dev/null Thu Jan 01 00:00:00 1970 +0000
68.2 +++ b/OpenSecurity/install/web.py-0.37/web/http.py Mon Dec 02 14:02:05 2013 +0100
68.3 @@ -0,0 +1,150 @@
68.4 +"""
68.5 +HTTP Utilities
68.6 +(from web.py)
68.7 +"""
68.8 +
68.9 +__all__ = [
68.10 + "expires", "lastmodified",
68.11 + "prefixurl", "modified",
68.12 + "changequery", "url",
68.13 + "profiler",
68.14 +]
68.15 +
68.16 +import sys, os, threading, urllib, urlparse
68.17 +try: import datetime
68.18 +except ImportError: pass
68.19 +import net, utils, webapi as web
68.20 +
68.21 +def prefixurl(base=''):
68.22 + """
68.23 + Sorry, this function is really difficult to explain.
68.24 + Maybe some other time.
68.25 + """
68.26 + url = web.ctx.path.lstrip('/')
68.27 + for i in xrange(url.count('/')):
68.28 + base += '../'
68.29 + if not base:
68.30 + base = './'
68.31 + return base
68.32 +
68.33 +def expires(delta):
68.34 + """
68.35 + Outputs an `Expires` header for `delta` from now.
68.36 + `delta` is a `timedelta` object or a number of seconds.
68.37 + """
68.38 + if isinstance(delta, (int, long)):
68.39 + delta = datetime.timedelta(seconds=delta)
68.40 + date_obj = datetime.datetime.utcnow() + delta
68.41 + web.header('Expires', net.httpdate(date_obj))
68.42 +
68.43 +def lastmodified(date_obj):
68.44 + """Outputs a `Last-Modified` header for `datetime`."""
68.45 + web.header('Last-Modified', net.httpdate(date_obj))
68.46 +
68.47 +def modified(date=None, etag=None):
68.48 + """
68.49 + Checks to see if the page has been modified since the version in the
68.50 + requester's cache.
68.51 +
68.52 + When you publish pages, you can include `Last-Modified` and `ETag`
68.53 + with the date the page was last modified and an opaque token for
68.54 + the particular version, respectively. When readers reload the page,
68.55 + the browser sends along the modification date and etag value for
68.56 + the version it has in its cache. If the page hasn't changed,
68.57 + the server can just return `304 Not Modified` and not have to
68.58 + send the whole page again.
68.59 +
68.60 + This function takes the last-modified date `date` and the ETag `etag`
68.61 + and checks the headers to see if they match. If they do, it returns
68.62 + `True`, or otherwise it raises NotModified error. It also sets
68.63 + `Last-Modified` and `ETag` output headers.
68.64 + """
68.65 + try:
68.66 + from __builtin__ import set
68.67 + except ImportError:
68.68 + # for python 2.3
68.69 + from sets import Set as set
68.70 +
68.71 + n = set([x.strip('" ') for x in web.ctx.env.get('HTTP_IF_NONE_MATCH', '').split(',')])
68.72 + m = net.parsehttpdate(web.ctx.env.get('HTTP_IF_MODIFIED_SINCE', '').split(';')[0])
68.73 + validate = False
68.74 + if etag:
68.75 + if '*' in n or etag in n:
68.76 + validate = True
68.77 + if date and m:
68.78 + # we subtract a second because
68.79 + # HTTP dates don't have sub-second precision
68.80 + if date-datetime.timedelta(seconds=1) <= m:
68.81 + validate = True
68.82 +
68.83 + if date: lastmodified(date)
68.84 + if etag: web.header('ETag', '"' + etag + '"')
68.85 + if validate:
68.86 + raise web.notmodified()
68.87 + else:
68.88 + return True
68.89 +
68.90 +def urlencode(query, doseq=0):
68.91 + """
68.92 + Same as urllib.urlencode, but supports unicode strings.
68.93 +
68.94 + >>> urlencode({'text':'foo bar'})
68.95 + 'text=foo+bar'
68.96 + >>> urlencode({'x': [1, 2]}, doseq=True)
68.97 + 'x=1&x=2'
68.98 + """
68.99 + def convert(value, doseq=False):
68.100 + if doseq and isinstance(value, list):
68.101 + return [convert(v) for v in value]
68.102 + else:
68.103 + return utils.safestr(value)
68.104 +
68.105 + query = dict([(k, convert(v, doseq)) for k, v in query.items()])
68.106 + return urllib.urlencode(query, doseq=doseq)
68.107 +
68.108 +def changequery(query=None, **kw):
68.109 + """
68.110 + Imagine you're at `/foo?a=1&b=2`. Then `changequery(a=3)` will return
68.111 + `/foo?a=3&b=2` -- the same URL but with the arguments you requested
68.112 + changed.
68.113 + """
68.114 + if query is None:
68.115 + query = web.rawinput(method='get')
68.116 + for k, v in kw.iteritems():
68.117 + if v is None:
68.118 + query.pop(k, None)
68.119 + else:
68.120 + query[k] = v
68.121 + out = web.ctx.path
68.122 + if query:
68.123 + out += '?' + urlencode(query, doseq=True)
68.124 + return out
68.125 +
68.126 +def url(path=None, doseq=False, **kw):
68.127 + """
68.128 + Makes url by concatenating web.ctx.homepath and path and the
68.129 + query string created using the arguments.
68.130 + """
68.131 + if path is None:
68.132 + path = web.ctx.path
68.133 + if path.startswith("/"):
68.134 + out = web.ctx.homepath + path
68.135 + else:
68.136 + out = path
68.137 +
68.138 + if kw:
68.139 + out += '?' + urlencode(kw, doseq=doseq)
68.140 +
68.141 + return out
68.142 +
68.143 +def profiler(app):
68.144 + """Outputs basic profiling information at the bottom of each response."""
68.145 + from utils import profile
68.146 + def profile_internal(e, o):
68.147 + out, result = profile(app)(e, o)
68.148 + return list(out) + ['<pre>' + net.websafe(result) + '</pre>']
68.149 + return profile_internal
68.150 +
68.151 +if __name__ == "__main__":
68.152 + import doctest
68.153 + doctest.testmod()
69.1 Binary file OpenSecurity/install/web.py-0.37/web/http.pyc has changed
70.1 --- /dev/null Thu Jan 01 00:00:00 1970 +0000
70.2 +++ b/OpenSecurity/install/web.py-0.37/web/httpserver.py Mon Dec 02 14:02:05 2013 +0100
70.3 @@ -0,0 +1,319 @@
70.4 +__all__ = ["runsimple"]
70.5 +
70.6 +import sys, os
70.7 +from SimpleHTTPServer import SimpleHTTPRequestHandler
70.8 +import urllib
70.9 +import posixpath
70.10 +
70.11 +import webapi as web
70.12 +import net
70.13 +import utils
70.14 +
70.15 +def runbasic(func, server_address=("0.0.0.0", 8080)):
70.16 + """
70.17 + Runs a simple HTTP server hosting WSGI app `func`. The directory `static/`
70.18 + is hosted statically.
70.19 +
70.20 + Based on [WsgiServer][ws] from [Colin Stewart][cs].
70.21 +
70.22 + [ws]: http://www.owlfish.com/software/wsgiutils/documentation/wsgi-server-api.html
70.23 + [cs]: http://www.owlfish.com/
70.24 + """
70.25 + # Copyright (c) 2004 Colin Stewart (http://www.owlfish.com/)
70.26 + # Modified somewhat for simplicity
70.27 + # Used under the modified BSD license:
70.28 + # http://www.xfree86.org/3.3.6/COPYRIGHT2.html#5
70.29 +
70.30 + import SimpleHTTPServer, SocketServer, BaseHTTPServer, urlparse
70.31 + import socket, errno
70.32 + import traceback
70.33 +
70.34 + class WSGIHandler(SimpleHTTPServer.SimpleHTTPRequestHandler):
70.35 + def run_wsgi_app(self):
70.36 + protocol, host, path, parameters, query, fragment = \
70.37 + urlparse.urlparse('http://dummyhost%s' % self.path)
70.38 +
70.39 + # we only use path, query
70.40 + env = {'wsgi.version': (1, 0)
70.41 + ,'wsgi.url_scheme': 'http'
70.42 + ,'wsgi.input': self.rfile
70.43 + ,'wsgi.errors': sys.stderr
70.44 + ,'wsgi.multithread': 1
70.45 + ,'wsgi.multiprocess': 0
70.46 + ,'wsgi.run_once': 0
70.47 + ,'REQUEST_METHOD': self.command
70.48 + ,'REQUEST_URI': self.path
70.49 + ,'PATH_INFO': path
70.50 + ,'QUERY_STRING': query
70.51 + ,'CONTENT_TYPE': self.headers.get('Content-Type', '')
70.52 + ,'CONTENT_LENGTH': self.headers.get('Content-Length', '')
70.53 + ,'REMOTE_ADDR': self.client_address[0]
70.54 + ,'SERVER_NAME': self.server.server_address[0]
70.55 + ,'SERVER_PORT': str(self.server.server_address[1])
70.56 + ,'SERVER_PROTOCOL': self.request_version
70.57 + }
70.58 +
70.59 + for http_header, http_value in self.headers.items():
70.60 + env ['HTTP_%s' % http_header.replace('-', '_').upper()] = \
70.61 + http_value
70.62 +
70.63 + # Setup the state
70.64 + self.wsgi_sent_headers = 0
70.65 + self.wsgi_headers = []
70.66 +
70.67 + try:
70.68 + # We have there environment, now invoke the application
70.69 + result = self.server.app(env, self.wsgi_start_response)
70.70 + try:
70.71 + try:
70.72 + for data in result:
70.73 + if data:
70.74 + self.wsgi_write_data(data)
70.75 + finally:
70.76 + if hasattr(result, 'close'):
70.77 + result.close()
70.78 + except socket.error, socket_err:
70.79 + # Catch common network errors and suppress them
70.80 + if (socket_err.args[0] in \
70.81 + (errno.ECONNABORTED, errno.EPIPE)):
70.82 + return
70.83 + except socket.timeout, socket_timeout:
70.84 + return
70.85 + except:
70.86 + print >> web.debug, traceback.format_exc(),
70.87 +
70.88 + if (not self.wsgi_sent_headers):
70.89 + # We must write out something!
70.90 + self.wsgi_write_data(" ")
70.91 + return
70.92 +
70.93 + do_POST = run_wsgi_app
70.94 + do_PUT = run_wsgi_app
70.95 + do_DELETE = run_wsgi_app
70.96 +
70.97 + def do_GET(self):
70.98 + if self.path.startswith('/static/'):
70.99 + SimpleHTTPServer.SimpleHTTPRequestHandler.do_GET(self)
70.100 + else:
70.101 + self.run_wsgi_app()
70.102 +
70.103 + def wsgi_start_response(self, response_status, response_headers,
70.104 + exc_info=None):
70.105 + if (self.wsgi_sent_headers):
70.106 + raise Exception \
70.107 + ("Headers already sent and start_response called again!")
70.108 + # Should really take a copy to avoid changes in the application....
70.109 + self.wsgi_headers = (response_status, response_headers)
70.110 + return self.wsgi_write_data
70.111 +
70.112 + def wsgi_write_data(self, data):
70.113 + if (not self.wsgi_sent_headers):
70.114 + status, headers = self.wsgi_headers
70.115 + # Need to send header prior to data
70.116 + status_code = status[:status.find(' ')]
70.117 + status_msg = status[status.find(' ') + 1:]
70.118 + self.send_response(int(status_code), status_msg)
70.119 + for header, value in headers:
70.120 + self.send_header(header, value)
70.121 + self.end_headers()
70.122 + self.wsgi_sent_headers = 1
70.123 + # Send the data
70.124 + self.wfile.write(data)
70.125 +
70.126 + class WSGIServer(SocketServer.ThreadingMixIn, BaseHTTPServer.HTTPServer):
70.127 + def __init__(self, func, server_address):
70.128 + BaseHTTPServer.HTTPServer.__init__(self,
70.129 + server_address,
70.130 + WSGIHandler)
70.131 + self.app = func
70.132 + self.serverShuttingDown = 0
70.133 +
70.134 + print "http://%s:%d/" % server_address
70.135 + WSGIServer(func, server_address).serve_forever()
70.136 +
70.137 +# The WSGIServer instance.
70.138 +# Made global so that it can be stopped in embedded mode.
70.139 +server = None
70.140 +
70.141 +def runsimple(func, server_address=("0.0.0.0", 8080)):
70.142 + """
70.143 + Runs [CherryPy][cp] WSGI server hosting WSGI app `func`.
70.144 + The directory `static/` is hosted statically.
70.145 +
70.146 + [cp]: http://www.cherrypy.org
70.147 + """
70.148 + global server
70.149 + func = StaticMiddleware(func)
70.150 + func = LogMiddleware(func)
70.151 +
70.152 + server = WSGIServer(server_address, func)
70.153 +
70.154 + if server.ssl_adapter:
70.155 + print "https://%s:%d/" % server_address
70.156 + else:
70.157 + print "http://%s:%d/" % server_address
70.158 +
70.159 + try:
70.160 + server.start()
70.161 + except (KeyboardInterrupt, SystemExit):
70.162 + server.stop()
70.163 + server = None
70.164 +
70.165 +def WSGIServer(server_address, wsgi_app):
70.166 + """Creates CherryPy WSGI server listening at `server_address` to serve `wsgi_app`.
70.167 + This function can be overwritten to customize the webserver or use a different webserver.
70.168 + """
70.169 + import wsgiserver
70.170 +
70.171 + # Default values of wsgiserver.ssl_adapters uses cherrypy.wsgiserver
70.172 + # prefix. Overwriting it make it work with web.wsgiserver.
70.173 + wsgiserver.ssl_adapters = {
70.174 + 'builtin': 'web.wsgiserver.ssl_builtin.BuiltinSSLAdapter',
70.175 + 'pyopenssl': 'web.wsgiserver.ssl_pyopenssl.pyOpenSSLAdapter',
70.176 + }
70.177 +
70.178 + server = wsgiserver.CherryPyWSGIServer(server_address, wsgi_app, server_name="localhost")
70.179 +
70.180 + def create_ssl_adapter(cert, key):
70.181 + # wsgiserver tries to import submodules as cherrypy.wsgiserver.foo.
70.182 + # That doesn't work as not it is web.wsgiserver.
70.183 + # Patching sys.modules temporarily to make it work.
70.184 + import types
70.185 + cherrypy = types.ModuleType('cherrypy')
70.186 + cherrypy.wsgiserver = wsgiserver
70.187 + sys.modules['cherrypy'] = cherrypy
70.188 + sys.modules['cherrypy.wsgiserver'] = wsgiserver
70.189 +
70.190 + from wsgiserver.ssl_pyopenssl import pyOpenSSLAdapter
70.191 + adapter = pyOpenSSLAdapter(cert, key)
70.192 +
70.193 + # We are done with our work. Cleanup the patches.
70.194 + del sys.modules['cherrypy']
70.195 + del sys.modules['cherrypy.wsgiserver']
70.196 +
70.197 + return adapter
70.198 +
70.199 + # SSL backward compatibility
70.200 + if (server.ssl_adapter is None and
70.201 + getattr(server, 'ssl_certificate', None) and
70.202 + getattr(server, 'ssl_private_key', None)):
70.203 + server.ssl_adapter = create_ssl_adapter(server.ssl_certificate, server.ssl_private_key)
70.204 +
70.205 + server.nodelay = not sys.platform.startswith('java') # TCP_NODELAY isn't supported on the JVM
70.206 + return server
70.207 +
70.208 +class StaticApp(SimpleHTTPRequestHandler):
70.209 + """WSGI application for serving static files."""
70.210 + def __init__(self, environ, start_response):
70.211 + self.headers = []
70.212 + self.environ = environ
70.213 + self.start_response = start_response
70.214 +
70.215 + def send_response(self, status, msg=""):
70.216 + self.status = str(status) + " " + msg
70.217 +
70.218 + def send_header(self, name, value):
70.219 + self.headers.append((name, value))
70.220 +
70.221 + def end_headers(self):
70.222 + pass
70.223 +
70.224 + def log_message(*a): pass
70.225 +
70.226 + def __iter__(self):
70.227 + environ = self.environ
70.228 +
70.229 + self.path = environ.get('PATH_INFO', '')
70.230 + self.client_address = environ.get('REMOTE_ADDR','-'), \
70.231 + environ.get('REMOTE_PORT','-')
70.232 + self.command = environ.get('REQUEST_METHOD', '-')
70.233 +
70.234 + from cStringIO import StringIO
70.235 + self.wfile = StringIO() # for capturing error
70.236 +
70.237 + try:
70.238 + path = self.translate_path(self.path)
70.239 + etag = '"%s"' % os.path.getmtime(path)
70.240 + client_etag = environ.get('HTTP_IF_NONE_MATCH')
70.241 + self.send_header('ETag', etag)
70.242 + if etag == client_etag:
70.243 + self.send_response(304, "Not Modified")
70.244 + self.start_response(self.status, self.headers)
70.245 + raise StopIteration
70.246 + except OSError:
70.247 + pass # Probably a 404
70.248 +
70.249 + f = self.send_head()
70.250 + self.start_response(self.status, self.headers)
70.251 +
70.252 + if f:
70.253 + block_size = 16 * 1024
70.254 + while True:
70.255 + buf = f.read(block_size)
70.256 + if not buf:
70.257 + break
70.258 + yield buf
70.259 + f.close()
70.260 + else:
70.261 + value = self.wfile.getvalue()
70.262 + yield value
70.263 +
70.264 +class StaticMiddleware:
70.265 + """WSGI middleware for serving static files."""
70.266 + def __init__(self, app, prefix='/static/'):
70.267 + self.app = app
70.268 + self.prefix = prefix
70.269 +
70.270 + def __call__(self, environ, start_response):
70.271 + path = environ.get('PATH_INFO', '')
70.272 + path = self.normpath(path)
70.273 +
70.274 + if path.startswith(self.prefix):
70.275 + return StaticApp(environ, start_response)
70.276 + else:
70.277 + return self.app(environ, start_response)
70.278 +
70.279 + def normpath(self, path):
70.280 + path2 = posixpath.normpath(urllib.unquote(path))
70.281 + if path.endswith("/"):
70.282 + path2 += "/"
70.283 + return path2
70.284 +
70.285 +
70.286 +class LogMiddleware:
70.287 + """WSGI middleware for logging the status."""
70.288 + def __init__(self, app):
70.289 + self.app = app
70.290 + self.format = '%s - - [%s] "%s %s %s" - %s'
70.291 +
70.292 + from BaseHTTPServer import BaseHTTPRequestHandler
70.293 + import StringIO
70.294 + f = StringIO.StringIO()
70.295 +
70.296 + class FakeSocket:
70.297 + def makefile(self, *a):
70.298 + return f
70.299 +
70.300 + # take log_date_time_string method from BaseHTTPRequestHandler
70.301 + self.log_date_time_string = BaseHTTPRequestHandler(FakeSocket(), None, None).log_date_time_string
70.302 +
70.303 + def __call__(self, environ, start_response):
70.304 + def xstart_response(status, response_headers, *args):
70.305 + out = start_response(status, response_headers, *args)
70.306 + self.log(status, environ)
70.307 + return out
70.308 +
70.309 + return self.app(environ, xstart_response)
70.310 +
70.311 + def log(self, status, environ):
70.312 + outfile = environ.get('wsgi.errors', web.debug)
70.313 + req = environ.get('PATH_INFO', '_')
70.314 + protocol = environ.get('ACTUAL_SERVER_PROTOCOL', '-')
70.315 + method = environ.get('REQUEST_METHOD', '-')
70.316 + host = "%s:%s" % (environ.get('REMOTE_ADDR','-'),
70.317 + environ.get('REMOTE_PORT','-'))
70.318 +
70.319 + time = self.log_date_time_string()
70.320 +
70.321 + msg = self.format % (host, time, protocol, method, req, status)
70.322 + print >> outfile, utils.safestr(msg)
71.1 Binary file OpenSecurity/install/web.py-0.37/web/httpserver.pyc has changed
72.1 --- /dev/null Thu Jan 01 00:00:00 1970 +0000
72.2 +++ b/OpenSecurity/install/web.py-0.37/web/net.py Mon Dec 02 14:02:05 2013 +0100
72.3 @@ -0,0 +1,193 @@
72.4 +"""
72.5 +Network Utilities
72.6 +(from web.py)
72.7 +"""
72.8 +
72.9 +__all__ = [
72.10 + "validipaddr", "validipport", "validip", "validaddr",
72.11 + "urlquote",
72.12 + "httpdate", "parsehttpdate",
72.13 + "htmlquote", "htmlunquote", "websafe",
72.14 +]
72.15 +
72.16 +import urllib, time
72.17 +try: import datetime
72.18 +except ImportError: pass
72.19 +
72.20 +def validipaddr(address):
72.21 + """
72.22 + Returns True if `address` is a valid IPv4 address.
72.23 +
72.24 + >>> validipaddr('192.168.1.1')
72.25 + True
72.26 + >>> validipaddr('192.168.1.800')
72.27 + False
72.28 + >>> validipaddr('192.168.1')
72.29 + False
72.30 + """
72.31 + try:
72.32 + octets = address.split('.')
72.33 + if len(octets) != 4:
72.34 + return False
72.35 + for x in octets:
72.36 + if not (0 <= int(x) <= 255):
72.37 + return False
72.38 + except ValueError:
72.39 + return False
72.40 + return True
72.41 +
72.42 +def validipport(port):
72.43 + """
72.44 + Returns True if `port` is a valid IPv4 port.
72.45 +
72.46 + >>> validipport('9000')
72.47 + True
72.48 + >>> validipport('foo')
72.49 + False
72.50 + >>> validipport('1000000')
72.51 + False
72.52 + """
72.53 + try:
72.54 + if not (0 <= int(port) <= 65535):
72.55 + return False
72.56 + except ValueError:
72.57 + return False
72.58 + return True
72.59 +
72.60 +def validip(ip, defaultaddr="0.0.0.0", defaultport=8080):
72.61 + """Returns `(ip_address, port)` from string `ip_addr_port`"""
72.62 + addr = defaultaddr
72.63 + port = defaultport
72.64 +
72.65 + ip = ip.split(":", 1)
72.66 + if len(ip) == 1:
72.67 + if not ip[0]:
72.68 + pass
72.69 + elif validipaddr(ip[0]):
72.70 + addr = ip[0]
72.71 + elif validipport(ip[0]):
72.72 + port = int(ip[0])
72.73 + else:
72.74 + raise ValueError, ':'.join(ip) + ' is not a valid IP address/port'
72.75 + elif len(ip) == 2:
72.76 + addr, port = ip
72.77 + if not validipaddr(addr) and validipport(port):
72.78 + raise ValueError, ':'.join(ip) + ' is not a valid IP address/port'
72.79 + port = int(port)
72.80 + else:
72.81 + raise ValueError, ':'.join(ip) + ' is not a valid IP address/port'
72.82 + return (addr, port)
72.83 +
72.84 +def validaddr(string_):
72.85 + """
72.86 + Returns either (ip_address, port) or "/path/to/socket" from string_
72.87 +
72.88 + >>> validaddr('/path/to/socket')
72.89 + '/path/to/socket'
72.90 + >>> validaddr('8000')
72.91 + ('0.0.0.0', 8000)
72.92 + >>> validaddr('127.0.0.1')
72.93 + ('127.0.0.1', 8080)
72.94 + >>> validaddr('127.0.0.1:8000')
72.95 + ('127.0.0.1', 8000)
72.96 + >>> validaddr('fff')
72.97 + Traceback (most recent call last):
72.98 + ...
72.99 + ValueError: fff is not a valid IP address/port
72.100 + """
72.101 + if '/' in string_:
72.102 + return string_
72.103 + else:
72.104 + return validip(string_)
72.105 +
72.106 +def urlquote(val):
72.107 + """
72.108 + Quotes a string for use in a URL.
72.109 +
72.110 + >>> urlquote('://?f=1&j=1')
72.111 + '%3A//%3Ff%3D1%26j%3D1'
72.112 + >>> urlquote(None)
72.113 + ''
72.114 + >>> urlquote(u'\u203d')
72.115 + '%E2%80%BD'
72.116 + """
72.117 + if val is None: return ''
72.118 + if not isinstance(val, unicode): val = str(val)
72.119 + else: val = val.encode('utf-8')
72.120 + return urllib.quote(val)
72.121 +
72.122 +def httpdate(date_obj):
72.123 + """
72.124 + Formats a datetime object for use in HTTP headers.
72.125 +
72.126 + >>> import datetime
72.127 + >>> httpdate(datetime.datetime(1970, 1, 1, 1, 1, 1))
72.128 + 'Thu, 01 Jan 1970 01:01:01 GMT'
72.129 + """
72.130 + return date_obj.strftime("%a, %d %b %Y %H:%M:%S GMT")
72.131 +
72.132 +def parsehttpdate(string_):
72.133 + """
72.134 + Parses an HTTP date into a datetime object.
72.135 +
72.136 + >>> parsehttpdate('Thu, 01 Jan 1970 01:01:01 GMT')
72.137 + datetime.datetime(1970, 1, 1, 1, 1, 1)
72.138 + """
72.139 + try:
72.140 + t = time.strptime(string_, "%a, %d %b %Y %H:%M:%S %Z")
72.141 + except ValueError:
72.142 + return None
72.143 + return datetime.datetime(*t[:6])
72.144 +
72.145 +def htmlquote(text):
72.146 + r"""
72.147 + Encodes `text` for raw use in HTML.
72.148 +
72.149 + >>> htmlquote(u"<'&\">")
72.150 + u'<'&">'
72.151 + """
72.152 + text = text.replace(u"&", u"&") # Must be done first!
72.153 + text = text.replace(u"<", u"<")
72.154 + text = text.replace(u">", u">")
72.155 + text = text.replace(u"'", u"'")
72.156 + text = text.replace(u'"', u""")
72.157 + return text
72.158 +
72.159 +def htmlunquote(text):
72.160 + r"""
72.161 + Decodes `text` that's HTML quoted.
72.162 +
72.163 + >>> htmlunquote(u'<'&">')
72.164 + u'<\'&">'
72.165 + """
72.166 + text = text.replace(u""", u'"')
72.167 + text = text.replace(u"'", u"'")
72.168 + text = text.replace(u">", u">")
72.169 + text = text.replace(u"<", u"<")
72.170 + text = text.replace(u"&", u"&") # Must be done last!
72.171 + return text
72.172 +
72.173 +def websafe(val):
72.174 + r"""Converts `val` so that it is safe for use in Unicode HTML.
72.175 +
72.176 + >>> websafe("<'&\">")
72.177 + u'<'&">'
72.178 + >>> websafe(None)
72.179 + u''
72.180 + >>> websafe(u'\u203d')
72.181 + u'\u203d'
72.182 + >>> websafe('\xe2\x80\xbd')
72.183 + u'\u203d'
72.184 + """
72.185 + if val is None:
72.186 + return u''
72.187 + elif isinstance(val, str):
72.188 + val = val.decode('utf-8')
72.189 + elif not isinstance(val, unicode):
72.190 + val = unicode(val)
72.191 +
72.192 + return htmlquote(val)
72.193 +
72.194 +if __name__ == "__main__":
72.195 + import doctest
72.196 + doctest.testmod()
73.1 Binary file OpenSecurity/install/web.py-0.37/web/net.pyc has changed
74.1 --- /dev/null Thu Jan 01 00:00:00 1970 +0000
74.2 +++ b/OpenSecurity/install/web.py-0.37/web/python23.py Mon Dec 02 14:02:05 2013 +0100
74.3 @@ -0,0 +1,46 @@
74.4 +"""Python 2.3 compatabilty"""
74.5 +import threading
74.6 +
74.7 +class threadlocal(object):
74.8 + """Implementation of threading.local for python2.3.
74.9 + """
74.10 + def __getattribute__(self, name):
74.11 + if name == "__dict__":
74.12 + return threadlocal._getd(self)
74.13 + else:
74.14 + try:
74.15 + return object.__getattribute__(self, name)
74.16 + except AttributeError:
74.17 + try:
74.18 + return self.__dict__[name]
74.19 + except KeyError:
74.20 + raise AttributeError, name
74.21 +
74.22 + def __setattr__(self, name, value):
74.23 + self.__dict__[name] = value
74.24 +
74.25 + def __delattr__(self, name):
74.26 + try:
74.27 + del self.__dict__[name]
74.28 + except KeyError:
74.29 + raise AttributeError, name
74.30 +
74.31 + def _getd(self):
74.32 + t = threading.currentThread()
74.33 + if not hasattr(t, '_d'):
74.34 + # using __dict__ of thread as thread local storage
74.35 + t._d = {}
74.36 +
74.37 + _id = id(self)
74.38 + # there could be multiple instances of threadlocal.
74.39 + # use id(self) as key
74.40 + if _id not in t._d:
74.41 + t._d[_id] = {}
74.42 + return t._d[_id]
74.43 +
74.44 +if __name__ == '__main__':
74.45 + d = threadlocal()
74.46 + d.x = 1
74.47 + print d.__dict__
74.48 + print d.x
74.49 +
74.50 \ No newline at end of file
75.1 --- /dev/null Thu Jan 01 00:00:00 1970 +0000
75.2 +++ b/OpenSecurity/install/web.py-0.37/web/session.py Mon Dec 02 14:02:05 2013 +0100
75.3 @@ -0,0 +1,358 @@
75.4 +"""
75.5 +Session Management
75.6 +(from web.py)
75.7 +"""
75.8 +
75.9 +import os, time, datetime, random, base64
75.10 +import os.path
75.11 +from copy import deepcopy
75.12 +try:
75.13 + import cPickle as pickle
75.14 +except ImportError:
75.15 + import pickle
75.16 +try:
75.17 + import hashlib
75.18 + sha1 = hashlib.sha1
75.19 +except ImportError:
75.20 + import sha
75.21 + sha1 = sha.new
75.22 +
75.23 +import utils
75.24 +import webapi as web
75.25 +
75.26 +__all__ = [
75.27 + 'Session', 'SessionExpired',
75.28 + 'Store', 'DiskStore', 'DBStore',
75.29 +]
75.30 +
75.31 +web.config.session_parameters = utils.storage({
75.32 + 'cookie_name': 'webpy_session_id',
75.33 + 'cookie_domain': None,
75.34 + 'cookie_path' : None,
75.35 + 'timeout': 86400, #24 * 60 * 60, # 24 hours in seconds
75.36 + 'ignore_expiry': True,
75.37 + 'ignore_change_ip': True,
75.38 + 'secret_key': 'fLjUfxqXtfNoIldA0A0J',
75.39 + 'expired_message': 'Session expired',
75.40 + 'httponly': True,
75.41 + 'secure': False
75.42 +})
75.43 +
75.44 +class SessionExpired(web.HTTPError):
75.45 + def __init__(self, message):
75.46 + web.HTTPError.__init__(self, '200 OK', {}, data=message)
75.47 +
75.48 +class Session(object):
75.49 + """Session management for web.py
75.50 + """
75.51 + __slots__ = [
75.52 + "store", "_initializer", "_last_cleanup_time", "_config", "_data",
75.53 + "__getitem__", "__setitem__", "__delitem__"
75.54 + ]
75.55 +
75.56 + def __init__(self, app, store, initializer=None):
75.57 + self.store = store
75.58 + self._initializer = initializer
75.59 + self._last_cleanup_time = 0
75.60 + self._config = utils.storage(web.config.session_parameters)
75.61 + self._data = utils.threadeddict()
75.62 +
75.63 + self.__getitem__ = self._data.__getitem__
75.64 + self.__setitem__ = self._data.__setitem__
75.65 + self.__delitem__ = self._data.__delitem__
75.66 +
75.67 + if app:
75.68 + app.add_processor(self._processor)
75.69 +
75.70 + def __contains__(self, name):
75.71 + return name in self._data
75.72 +
75.73 + def __getattr__(self, name):
75.74 + return getattr(self._data, name)
75.75 +
75.76 + def __setattr__(self, name, value):
75.77 + if name in self.__slots__:
75.78 + object.__setattr__(self, name, value)
75.79 + else:
75.80 + setattr(self._data, name, value)
75.81 +
75.82 + def __delattr__(self, name):
75.83 + delattr(self._data, name)
75.84 +
75.85 + def _processor(self, handler):
75.86 + """Application processor to setup session for every request"""
75.87 + self._cleanup()
75.88 + self._load()
75.89 +
75.90 + try:
75.91 + return handler()
75.92 + finally:
75.93 + self._save()
75.94 +
75.95 + def _load(self):
75.96 + """Load the session from the store, by the id from cookie"""
75.97 + cookie_name = self._config.cookie_name
75.98 + cookie_domain = self._config.cookie_domain
75.99 + cookie_path = self._config.cookie_path
75.100 + httponly = self._config.httponly
75.101 + self.session_id = web.cookies().get(cookie_name)
75.102 +
75.103 + # protection against session_id tampering
75.104 + if self.session_id and not self._valid_session_id(self.session_id):
75.105 + self.session_id = None
75.106 +
75.107 + self._check_expiry()
75.108 + if self.session_id:
75.109 + d = self.store[self.session_id]
75.110 + self.update(d)
75.111 + self._validate_ip()
75.112 +
75.113 + if not self.session_id:
75.114 + self.session_id = self._generate_session_id()
75.115 +
75.116 + if self._initializer:
75.117 + if isinstance(self._initializer, dict):
75.118 + self.update(deepcopy(self._initializer))
75.119 + elif hasattr(self._initializer, '__call__'):
75.120 + self._initializer()
75.121 +
75.122 + self.ip = web.ctx.ip
75.123 +
75.124 + def _check_expiry(self):
75.125 + # check for expiry
75.126 + if self.session_id and self.session_id not in self.store:
75.127 + if self._config.ignore_expiry:
75.128 + self.session_id = None
75.129 + else:
75.130 + return self.expired()
75.131 +
75.132 + def _validate_ip(self):
75.133 + # check for change of IP
75.134 + if self.session_id and self.get('ip', None) != web.ctx.ip:
75.135 + if not self._config.ignore_change_ip:
75.136 + return self.expired()
75.137 +
75.138 + def _save(self):
75.139 + if not self.get('_killed'):
75.140 + self._setcookie(self.session_id)
75.141 + self.store[self.session_id] = dict(self._data)
75.142 + else:
75.143 + self._setcookie(self.session_id, expires=-1)
75.144 +
75.145 + def _setcookie(self, session_id, expires='', **kw):
75.146 + cookie_name = self._config.cookie_name
75.147 + cookie_domain = self._config.cookie_domain
75.148 + cookie_path = self._config.cookie_path
75.149 + httponly = self._config.httponly
75.150 + secure = self._config.secure
75.151 + web.setcookie(cookie_name, session_id, expires=expires, domain=cookie_domain, httponly=httponly, secure=secure, path=cookie_path)
75.152 +
75.153 + def _generate_session_id(self):
75.154 + """Generate a random id for session"""
75.155 +
75.156 + while True:
75.157 + rand = os.urandom(16)
75.158 + now = time.time()
75.159 + secret_key = self._config.secret_key
75.160 + session_id = sha1("%s%s%s%s" %(rand, now, utils.safestr(web.ctx.ip), secret_key))
75.161 + session_id = session_id.hexdigest()
75.162 + if session_id not in self.store:
75.163 + break
75.164 + return session_id
75.165 +
75.166 + def _valid_session_id(self, session_id):
75.167 + rx = utils.re_compile('^[0-9a-fA-F]+$')
75.168 + return rx.match(session_id)
75.169 +
75.170 + def _cleanup(self):
75.171 + """Cleanup the stored sessions"""
75.172 + current_time = time.time()
75.173 + timeout = self._config.timeout
75.174 + if current_time - self._last_cleanup_time > timeout:
75.175 + self.store.cleanup(timeout)
75.176 + self._last_cleanup_time = current_time
75.177 +
75.178 + def expired(self):
75.179 + """Called when an expired session is atime"""
75.180 + self._killed = True
75.181 + self._save()
75.182 + raise SessionExpired(self._config.expired_message)
75.183 +
75.184 + def kill(self):
75.185 + """Kill the session, make it no longer available"""
75.186 + del self.store[self.session_id]
75.187 + self._killed = True
75.188 +
75.189 +class Store:
75.190 + """Base class for session stores"""
75.191 +
75.192 + def __contains__(self, key):
75.193 + raise NotImplementedError
75.194 +
75.195 + def __getitem__(self, key):
75.196 + raise NotImplementedError
75.197 +
75.198 + def __setitem__(self, key, value):
75.199 + raise NotImplementedError
75.200 +
75.201 + def cleanup(self, timeout):
75.202 + """removes all the expired sessions"""
75.203 + raise NotImplementedError
75.204 +
75.205 + def encode(self, session_dict):
75.206 + """encodes session dict as a string"""
75.207 + pickled = pickle.dumps(session_dict)
75.208 + return base64.encodestring(pickled)
75.209 +
75.210 + def decode(self, session_data):
75.211 + """decodes the data to get back the session dict """
75.212 + pickled = base64.decodestring(session_data)
75.213 + return pickle.loads(pickled)
75.214 +
75.215 +class DiskStore(Store):
75.216 + """
75.217 + Store for saving a session on disk.
75.218 +
75.219 + >>> import tempfile
75.220 + >>> root = tempfile.mkdtemp()
75.221 + >>> s = DiskStore(root)
75.222 + >>> s['a'] = 'foo'
75.223 + >>> s['a']
75.224 + 'foo'
75.225 + >>> time.sleep(0.01)
75.226 + >>> s.cleanup(0.01)
75.227 + >>> s['a']
75.228 + Traceback (most recent call last):
75.229 + ...
75.230 + KeyError: 'a'
75.231 + """
75.232 + def __init__(self, root):
75.233 + # if the storage root doesn't exists, create it.
75.234 + if not os.path.exists(root):
75.235 + os.makedirs(
75.236 + os.path.abspath(root)
75.237 + )
75.238 + self.root = root
75.239 +
75.240 + def _get_path(self, key):
75.241 + if os.path.sep in key:
75.242 + raise ValueError, "Bad key: %s" % repr(key)
75.243 + return os.path.join(self.root, key)
75.244 +
75.245 + def __contains__(self, key):
75.246 + path = self._get_path(key)
75.247 + return os.path.exists(path)
75.248 +
75.249 + def __getitem__(self, key):
75.250 + path = self._get_path(key)
75.251 + if os.path.exists(path):
75.252 + pickled = open(path).read()
75.253 + return self.decode(pickled)
75.254 + else:
75.255 + raise KeyError, key
75.256 +
75.257 + def __setitem__(self, key, value):
75.258 + path = self._get_path(key)
75.259 + pickled = self.encode(value)
75.260 + try:
75.261 + f = open(path, 'w')
75.262 + try:
75.263 + f.write(pickled)
75.264 + finally:
75.265 + f.close()
75.266 + except IOError:
75.267 + pass
75.268 +
75.269 + def __delitem__(self, key):
75.270 + path = self._get_path(key)
75.271 + if os.path.exists(path):
75.272 + os.remove(path)
75.273 +
75.274 + def cleanup(self, timeout):
75.275 + now = time.time()
75.276 + for f in os.listdir(self.root):
75.277 + path = self._get_path(f)
75.278 + atime = os.stat(path).st_atime
75.279 + if now - atime > timeout :
75.280 + os.remove(path)
75.281 +
75.282 +class DBStore(Store):
75.283 + """Store for saving a session in database
75.284 + Needs a table with the following columns:
75.285 +
75.286 + session_id CHAR(128) UNIQUE NOT NULL,
75.287 + atime DATETIME NOT NULL default current_timestamp,
75.288 + data TEXT
75.289 + """
75.290 + def __init__(self, db, table_name):
75.291 + self.db = db
75.292 + self.table = table_name
75.293 +
75.294 + def __contains__(self, key):
75.295 + data = self.db.select(self.table, where="session_id=$key", vars=locals())
75.296 + return bool(list(data))
75.297 +
75.298 + def __getitem__(self, key):
75.299 + now = datetime.datetime.now()
75.300 + try:
75.301 + s = self.db.select(self.table, where="session_id=$key", vars=locals())[0]
75.302 + self.db.update(self.table, where="session_id=$key", atime=now, vars=locals())
75.303 + except IndexError:
75.304 + raise KeyError
75.305 + else:
75.306 + return self.decode(s.data)
75.307 +
75.308 + def __setitem__(self, key, value):
75.309 + pickled = self.encode(value)
75.310 + now = datetime.datetime.now()
75.311 + if key in self:
75.312 + self.db.update(self.table, where="session_id=$key", data=pickled, vars=locals())
75.313 + else:
75.314 + self.db.insert(self.table, False, session_id=key, data=pickled )
75.315 +
75.316 + def __delitem__(self, key):
75.317 + self.db.delete(self.table, where="session_id=$key", vars=locals())
75.318 +
75.319 + def cleanup(self, timeout):
75.320 + timeout = datetime.timedelta(timeout/(24.0*60*60)) #timedelta takes numdays as arg
75.321 + last_allowed_time = datetime.datetime.now() - timeout
75.322 + self.db.delete(self.table, where="$last_allowed_time > atime", vars=locals())
75.323 +
75.324 +class ShelfStore:
75.325 + """Store for saving session using `shelve` module.
75.326 +
75.327 + import shelve
75.328 + store = ShelfStore(shelve.open('session.shelf'))
75.329 +
75.330 + XXX: is shelve thread-safe?
75.331 + """
75.332 + def __init__(self, shelf):
75.333 + self.shelf = shelf
75.334 +
75.335 + def __contains__(self, key):
75.336 + return key in self.shelf
75.337 +
75.338 + def __getitem__(self, key):
75.339 + atime, v = self.shelf[key]
75.340 + self[key] = v # update atime
75.341 + return v
75.342 +
75.343 + def __setitem__(self, key, value):
75.344 + self.shelf[key] = time.time(), value
75.345 +
75.346 + def __delitem__(self, key):
75.347 + try:
75.348 + del self.shelf[key]
75.349 + except KeyError:
75.350 + pass
75.351 +
75.352 + def cleanup(self, timeout):
75.353 + now = time.time()
75.354 + for k in self.shelf.keys():
75.355 + atime, v = self.shelf[k]
75.356 + if now - atime > timeout :
75.357 + del self[k]
75.358 +
75.359 +if __name__ == '__main__' :
75.360 + import doctest
75.361 + doctest.testmod()
76.1 Binary file OpenSecurity/install/web.py-0.37/web/session.pyc has changed
77.1 --- /dev/null Thu Jan 01 00:00:00 1970 +0000
77.2 +++ b/OpenSecurity/install/web.py-0.37/web/template.py Mon Dec 02 14:02:05 2013 +0100
77.3 @@ -0,0 +1,1515 @@
77.4 +"""
77.5 +simple, elegant templating
77.6 +(part of web.py)
77.7 +
77.8 +Template design:
77.9 +
77.10 +Template string is split into tokens and the tokens are combined into nodes.
77.11 +Parse tree is a nodelist. TextNode and ExpressionNode are simple nodes and
77.12 +for-loop, if-loop etc are block nodes, which contain multiple child nodes.
77.13 +
77.14 +Each node can emit some python string. python string emitted by the
77.15 +root node is validated for safeeval and executed using python in the given environment.
77.16 +
77.17 +Enough care is taken to make sure the generated code and the template has line to line match,
77.18 +so that the error messages can point to exact line number in template. (It doesn't work in some cases still.)
77.19 +
77.20 +Grammar:
77.21 +
77.22 + template -> defwith sections
77.23 + defwith -> '$def with (' arguments ')' | ''
77.24 + sections -> section*
77.25 + section -> block | assignment | line
77.26 +
77.27 + assignment -> '$ ' <assignment expression>
77.28 + line -> (text|expr)*
77.29 + text -> <any characters other than $>
77.30 + expr -> '$' pyexpr | '$(' pyexpr ')' | '${' pyexpr '}'
77.31 + pyexpr -> <python expression>
77.32 +"""
77.33 +
77.34 +__all__ = [
77.35 + "Template",
77.36 + "Render", "render", "frender",
77.37 + "ParseError", "SecurityError",
77.38 + "test"
77.39 +]
77.40 +
77.41 +import tokenize
77.42 +import os
77.43 +import sys
77.44 +import glob
77.45 +import re
77.46 +from UserDict import DictMixin
77.47 +import warnings
77.48 +
77.49 +from utils import storage, safeunicode, safestr, re_compile
77.50 +from webapi import config
77.51 +from net import websafe
77.52 +
77.53 +def splitline(text):
77.54 + r"""
77.55 + Splits the given text at newline.
77.56 +
77.57 + >>> splitline('foo\nbar')
77.58 + ('foo\n', 'bar')
77.59 + >>> splitline('foo')
77.60 + ('foo', '')
77.61 + >>> splitline('')
77.62 + ('', '')
77.63 + """
77.64 + index = text.find('\n') + 1
77.65 + if index:
77.66 + return text[:index], text[index:]
77.67 + else:
77.68 + return text, ''
77.69 +
77.70 +class Parser:
77.71 + """Parser Base.
77.72 + """
77.73 + def __init__(self):
77.74 + self.statement_nodes = STATEMENT_NODES
77.75 + self.keywords = KEYWORDS
77.76 +
77.77 + def parse(self, text, name="<template>"):
77.78 + self.text = text
77.79 + self.name = name
77.80 +
77.81 + defwith, text = self.read_defwith(text)
77.82 + suite = self.read_suite(text)
77.83 + return DefwithNode(defwith, suite)
77.84 +
77.85 + def read_defwith(self, text):
77.86 + if text.startswith('$def with'):
77.87 + defwith, text = splitline(text)
77.88 + defwith = defwith[1:].strip() # strip $ and spaces
77.89 + return defwith, text
77.90 + else:
77.91 + return '', text
77.92 +
77.93 + def read_section(self, text):
77.94 + r"""Reads one section from the given text.
77.95 +
77.96 + section -> block | assignment | line
77.97 +
77.98 + >>> read_section = Parser().read_section
77.99 + >>> read_section('foo\nbar\n')
77.100 + (<line: [t'foo\n']>, 'bar\n')
77.101 + >>> read_section('$ a = b + 1\nfoo\n')
77.102 + (<assignment: 'a = b + 1'>, 'foo\n')
77.103 +
77.104 + read_section('$for in range(10):\n hello $i\nfoo)
77.105 + """
77.106 + if text.lstrip(' ').startswith('$'):
77.107 + index = text.index('$')
77.108 + begin_indent, text2 = text[:index], text[index+1:]
77.109 + ahead = self.python_lookahead(text2)
77.110 +
77.111 + if ahead == 'var':
77.112 + return self.read_var(text2)
77.113 + elif ahead in self.statement_nodes:
77.114 + return self.read_block_section(text2, begin_indent)
77.115 + elif ahead in self.keywords:
77.116 + return self.read_keyword(text2)
77.117 + elif ahead.strip() == '':
77.118 + # assignments starts with a space after $
77.119 + # ex: $ a = b + 2
77.120 + return self.read_assignment(text2)
77.121 + return self.readline(text)
77.122 +
77.123 + def read_var(self, text):
77.124 + r"""Reads a var statement.
77.125 +
77.126 + >>> read_var = Parser().read_var
77.127 + >>> read_var('var x=10\nfoo')
77.128 + (<var: x = 10>, 'foo')
77.129 + >>> read_var('var x: hello $name\nfoo')
77.130 + (<var: x = join_(u'hello ', escape_(name, True))>, 'foo')
77.131 + """
77.132 + line, text = splitline(text)
77.133 + tokens = self.python_tokens(line)
77.134 + if len(tokens) < 4:
77.135 + raise SyntaxError('Invalid var statement')
77.136 +
77.137 + name = tokens[1]
77.138 + sep = tokens[2]
77.139 + value = line.split(sep, 1)[1].strip()
77.140 +
77.141 + if sep == '=':
77.142 + pass # no need to process value
77.143 + elif sep == ':':
77.144 + #@@ Hack for backward-compatability
77.145 + if tokens[3] == '\n': # multi-line var statement
77.146 + block, text = self.read_indented_block(text, ' ')
77.147 + lines = [self.readline(x)[0] for x in block.splitlines()]
77.148 + nodes = []
77.149 + for x in lines:
77.150 + nodes.extend(x.nodes)
77.151 + nodes.append(TextNode('\n'))
77.152 + else: # single-line var statement
77.153 + linenode, _ = self.readline(value)
77.154 + nodes = linenode.nodes
77.155 + parts = [node.emit('') for node in nodes]
77.156 + value = "join_(%s)" % ", ".join(parts)
77.157 + else:
77.158 + raise SyntaxError('Invalid var statement')
77.159 + return VarNode(name, value), text
77.160 +
77.161 + def read_suite(self, text):
77.162 + r"""Reads section by section till end of text.
77.163 +
77.164 + >>> read_suite = Parser().read_suite
77.165 + >>> read_suite('hello $name\nfoo\n')
77.166 + [<line: [t'hello ', $name, t'\n']>, <line: [t'foo\n']>]
77.167 + """
77.168 + sections = []
77.169 + while text:
77.170 + section, text = self.read_section(text)
77.171 + sections.append(section)
77.172 + return SuiteNode(sections)
77.173 +
77.174 + def readline(self, text):
77.175 + r"""Reads one line from the text. Newline is supressed if the line ends with \.
77.176 +
77.177 + >>> readline = Parser().readline
77.178 + >>> readline('hello $name!\nbye!')
77.179 + (<line: [t'hello ', $name, t'!\n']>, 'bye!')
77.180 + >>> readline('hello $name!\\\nbye!')
77.181 + (<line: [t'hello ', $name, t'!']>, 'bye!')
77.182 + >>> readline('$f()\n\n')
77.183 + (<line: [$f(), t'\n']>, '\n')
77.184 + """
77.185 + line, text = splitline(text)
77.186 +
77.187 + # supress new line if line ends with \
77.188 + if line.endswith('\\\n'):
77.189 + line = line[:-2]
77.190 +
77.191 + nodes = []
77.192 + while line:
77.193 + node, line = self.read_node(line)
77.194 + nodes.append(node)
77.195 +
77.196 + return LineNode(nodes), text
77.197 +
77.198 + def read_node(self, text):
77.199 + r"""Reads a node from the given text and returns the node and remaining text.
77.200 +
77.201 + >>> read_node = Parser().read_node
77.202 + >>> read_node('hello $name')
77.203 + (t'hello ', '$name')
77.204 + >>> read_node('$name')
77.205 + ($name, '')
77.206 + """
77.207 + if text.startswith('$$'):
77.208 + return TextNode('$'), text[2:]
77.209 + elif text.startswith('$#'): # comment
77.210 + line, text = splitline(text)
77.211 + return TextNode('\n'), text
77.212 + elif text.startswith('$'):
77.213 + text = text[1:] # strip $
77.214 + if text.startswith(':'):
77.215 + escape = False
77.216 + text = text[1:] # strip :
77.217 + else:
77.218 + escape = True
77.219 + return self.read_expr(text, escape=escape)
77.220 + else:
77.221 + return self.read_text(text)
77.222 +
77.223 + def read_text(self, text):
77.224 + r"""Reads a text node from the given text.
77.225 +
77.226 + >>> read_text = Parser().read_text
77.227 + >>> read_text('hello $name')
77.228 + (t'hello ', '$name')
77.229 + """
77.230 + index = text.find('$')
77.231 + if index < 0:
77.232 + return TextNode(text), ''
77.233 + else:
77.234 + return TextNode(text[:index]), text[index:]
77.235 +
77.236 + def read_keyword(self, text):
77.237 + line, text = splitline(text)
77.238 + return StatementNode(line.strip() + "\n"), text
77.239 +
77.240 + def read_expr(self, text, escape=True):
77.241 + """Reads a python expression from the text and returns the expression and remaining text.
77.242 +
77.243 + expr -> simple_expr | paren_expr
77.244 + simple_expr -> id extended_expr
77.245 + extended_expr -> attr_access | paren_expr extended_expr | ''
77.246 + attr_access -> dot id extended_expr
77.247 + paren_expr -> [ tokens ] | ( tokens ) | { tokens }
77.248 +
77.249 + >>> read_expr = Parser().read_expr
77.250 + >>> read_expr("name")
77.251 + ($name, '')
77.252 + >>> read_expr("a.b and c")
77.253 + ($a.b, ' and c')
77.254 + >>> read_expr("a. b")
77.255 + ($a, '. b')
77.256 + >>> read_expr("name</h1>")
77.257 + ($name, '</h1>')
77.258 + >>> read_expr("(limit)ing")
77.259 + ($(limit), 'ing')
77.260 + >>> read_expr('a[1, 2][:3].f(1+2, "weird string[).", 3 + 4) done.')
77.261 + ($a[1, 2][:3].f(1+2, "weird string[).", 3 + 4), ' done.')
77.262 + """
77.263 + def simple_expr():
77.264 + identifier()
77.265 + extended_expr()
77.266 +
77.267 + def identifier():
77.268 + tokens.next()
77.269 +
77.270 + def extended_expr():
77.271 + lookahead = tokens.lookahead()
77.272 + if lookahead is None:
77.273 + return
77.274 + elif lookahead.value == '.':
77.275 + attr_access()
77.276 + elif lookahead.value in parens:
77.277 + paren_expr()
77.278 + extended_expr()
77.279 + else:
77.280 + return
77.281 +
77.282 + def attr_access():
77.283 + from token import NAME # python token constants
77.284 + dot = tokens.lookahead()
77.285 + if tokens.lookahead2().type == NAME:
77.286 + tokens.next() # consume dot
77.287 + identifier()
77.288 + extended_expr()
77.289 +
77.290 + def paren_expr():
77.291 + begin = tokens.next().value
77.292 + end = parens[begin]
77.293 + while True:
77.294 + if tokens.lookahead().value in parens:
77.295 + paren_expr()
77.296 + else:
77.297 + t = tokens.next()
77.298 + if t.value == end:
77.299 + break
77.300 + return
77.301 +
77.302 + parens = {
77.303 + "(": ")",
77.304 + "[": "]",
77.305 + "{": "}"
77.306 + }
77.307 +
77.308 + def get_tokens(text):
77.309 + """tokenize text using python tokenizer.
77.310 + Python tokenizer ignores spaces, but they might be important in some cases.
77.311 + This function introduces dummy space tokens when it identifies any ignored space.
77.312 + Each token is a storage object containing type, value, begin and end.
77.313 + """
77.314 + readline = iter([text]).next
77.315 + end = None
77.316 + for t in tokenize.generate_tokens(readline):
77.317 + t = storage(type=t[0], value=t[1], begin=t[2], end=t[3])
77.318 + if end is not None and end != t.begin:
77.319 + _, x1 = end
77.320 + _, x2 = t.begin
77.321 + yield storage(type=-1, value=text[x1:x2], begin=end, end=t.begin)
77.322 + end = t.end
77.323 + yield t
77.324 +
77.325 + class BetterIter:
77.326 + """Iterator like object with 2 support for 2 look aheads."""
77.327 + def __init__(self, items):
77.328 + self.iteritems = iter(items)
77.329 + self.items = []
77.330 + self.position = 0
77.331 + self.current_item = None
77.332 +
77.333 + def lookahead(self):
77.334 + if len(self.items) <= self.position:
77.335 + self.items.append(self._next())
77.336 + return self.items[self.position]
77.337 +
77.338 + def _next(self):
77.339 + try:
77.340 + return self.iteritems.next()
77.341 + except StopIteration:
77.342 + return None
77.343 +
77.344 + def lookahead2(self):
77.345 + if len(self.items) <= self.position+1:
77.346 + self.items.append(self._next())
77.347 + return self.items[self.position+1]
77.348 +
77.349 + def next(self):
77.350 + self.current_item = self.lookahead()
77.351 + self.position += 1
77.352 + return self.current_item
77.353 +
77.354 + tokens = BetterIter(get_tokens(text))
77.355 +
77.356 + if tokens.lookahead().value in parens:
77.357 + paren_expr()
77.358 + else:
77.359 + simple_expr()
77.360 + row, col = tokens.current_item.end
77.361 + return ExpressionNode(text[:col], escape=escape), text[col:]
77.362 +
77.363 + def read_assignment(self, text):
77.364 + r"""Reads assignment statement from text.
77.365 +
77.366 + >>> read_assignment = Parser().read_assignment
77.367 + >>> read_assignment('a = b + 1\nfoo')
77.368 + (<assignment: 'a = b + 1'>, 'foo')
77.369 + """
77.370 + line, text = splitline(text)
77.371 + return AssignmentNode(line.strip()), text
77.372 +
77.373 + def python_lookahead(self, text):
77.374 + """Returns the first python token from the given text.
77.375 +
77.376 + >>> python_lookahead = Parser().python_lookahead
77.377 + >>> python_lookahead('for i in range(10):')
77.378 + 'for'
77.379 + >>> python_lookahead('else:')
77.380 + 'else'
77.381 + >>> python_lookahead(' x = 1')
77.382 + ' '
77.383 + """
77.384 + readline = iter([text]).next
77.385 + tokens = tokenize.generate_tokens(readline)
77.386 + return tokens.next()[1]
77.387 +
77.388 + def python_tokens(self, text):
77.389 + readline = iter([text]).next
77.390 + tokens = tokenize.generate_tokens(readline)
77.391 + return [t[1] for t in tokens]
77.392 +
77.393 + def read_indented_block(self, text, indent):
77.394 + r"""Read a block of text. A block is what typically follows a for or it statement.
77.395 + It can be in the same line as that of the statement or an indented block.
77.396 +
77.397 + >>> read_indented_block = Parser().read_indented_block
77.398 + >>> read_indented_block(' a\n b\nc', ' ')
77.399 + ('a\nb\n', 'c')
77.400 + >>> read_indented_block(' a\n b\n c\nd', ' ')
77.401 + ('a\n b\nc\n', 'd')
77.402 + >>> read_indented_block(' a\n\n b\nc', ' ')
77.403 + ('a\n\n b\n', 'c')
77.404 + """
77.405 + if indent == '':
77.406 + return '', text
77.407 +
77.408 + block = ""
77.409 + while text:
77.410 + line, text2 = splitline(text)
77.411 + if line.strip() == "":
77.412 + block += '\n'
77.413 + elif line.startswith(indent):
77.414 + block += line[len(indent):]
77.415 + else:
77.416 + break
77.417 + text = text2
77.418 + return block, text
77.419 +
77.420 + def read_statement(self, text):
77.421 + r"""Reads a python statement.
77.422 +
77.423 + >>> read_statement = Parser().read_statement
77.424 + >>> read_statement('for i in range(10): hello $name')
77.425 + ('for i in range(10):', ' hello $name')
77.426 + """
77.427 + tok = PythonTokenizer(text)
77.428 + tok.consume_till(':')
77.429 + return text[:tok.index], text[tok.index:]
77.430 +
77.431 + def read_block_section(self, text, begin_indent=''):
77.432 + r"""
77.433 + >>> read_block_section = Parser().read_block_section
77.434 + >>> read_block_section('for i in range(10): hello $i\nfoo')
77.435 + (<block: 'for i in range(10):', [<line: [t'hello ', $i, t'\n']>]>, 'foo')
77.436 + >>> read_block_section('for i in range(10):\n hello $i\n foo', begin_indent=' ')
77.437 + (<block: 'for i in range(10):', [<line: [t'hello ', $i, t'\n']>]>, ' foo')
77.438 + >>> read_block_section('for i in range(10):\n hello $i\nfoo')
77.439 + (<block: 'for i in range(10):', [<line: [t'hello ', $i, t'\n']>]>, 'foo')
77.440 + """
77.441 + line, text = splitline(text)
77.442 + stmt, line = self.read_statement(line)
77.443 + keyword = self.python_lookahead(stmt)
77.444 +
77.445 + # if there is some thing left in the line
77.446 + if line.strip():
77.447 + block = line.lstrip()
77.448 + else:
77.449 + def find_indent(text):
77.450 + rx = re_compile(' +')
77.451 + match = rx.match(text)
77.452 + first_indent = match and match.group(0)
77.453 + return first_indent or ""
77.454 +
77.455 + # find the indentation of the block by looking at the first line
77.456 + first_indent = find_indent(text)[len(begin_indent):]
77.457 +
77.458 + #TODO: fix this special case
77.459 + if keyword == "code":
77.460 + indent = begin_indent + first_indent
77.461 + else:
77.462 + indent = begin_indent + min(first_indent, INDENT)
77.463 +
77.464 + block, text = self.read_indented_block(text, indent)
77.465 +
77.466 + return self.create_block_node(keyword, stmt, block, begin_indent), text
77.467 +
77.468 + def create_block_node(self, keyword, stmt, block, begin_indent):
77.469 + if keyword in self.statement_nodes:
77.470 + return self.statement_nodes[keyword](stmt, block, begin_indent)
77.471 + else:
77.472 + raise ParseError, 'Unknown statement: %s' % repr(keyword)
77.473 +
77.474 +class PythonTokenizer:
77.475 + """Utility wrapper over python tokenizer."""
77.476 + def __init__(self, text):
77.477 + self.text = text
77.478 + readline = iter([text]).next
77.479 + self.tokens = tokenize.generate_tokens(readline)
77.480 + self.index = 0
77.481 +
77.482 + def consume_till(self, delim):
77.483 + """Consumes tokens till colon.
77.484 +
77.485 + >>> tok = PythonTokenizer('for i in range(10): hello $i')
77.486 + >>> tok.consume_till(':')
77.487 + >>> tok.text[:tok.index]
77.488 + 'for i in range(10):'
77.489 + >>> tok.text[tok.index:]
77.490 + ' hello $i'
77.491 + """
77.492 + try:
77.493 + while True:
77.494 + t = self.next()
77.495 + if t.value == delim:
77.496 + break
77.497 + elif t.value == '(':
77.498 + self.consume_till(')')
77.499 + elif t.value == '[':
77.500 + self.consume_till(']')
77.501 + elif t.value == '{':
77.502 + self.consume_till('}')
77.503 +
77.504 + # if end of line is found, it is an exception.
77.505 + # Since there is no easy way to report the line number,
77.506 + # leave the error reporting to the python parser later
77.507 + #@@ This should be fixed.
77.508 + if t.value == '\n':
77.509 + break
77.510 + except:
77.511 + #raise ParseError, "Expected %s, found end of line." % repr(delim)
77.512 +
77.513 + # raising ParseError doesn't show the line number.
77.514 + # if this error is ignored, then it will be caught when compiling the python code.
77.515 + return
77.516 +
77.517 + def next(self):
77.518 + type, t, begin, end, line = self.tokens.next()
77.519 + row, col = end
77.520 + self.index = col
77.521 + return storage(type=type, value=t, begin=begin, end=end)
77.522 +
77.523 +class DefwithNode:
77.524 + def __init__(self, defwith, suite):
77.525 + if defwith:
77.526 + self.defwith = defwith.replace('with', '__template__') + ':'
77.527 + # offset 4 lines. for encoding, __lineoffset__, loop and self.
77.528 + self.defwith += "\n __lineoffset__ = -4"
77.529 + else:
77.530 + self.defwith = 'def __template__():'
77.531 + # offset 4 lines for encoding, __template__, __lineoffset__, loop and self.
77.532 + self.defwith += "\n __lineoffset__ = -5"
77.533 +
77.534 + self.defwith += "\n loop = ForLoop()"
77.535 + self.defwith += "\n self = TemplateResult(); extend_ = self.extend"
77.536 + self.suite = suite
77.537 + self.end = "\n return self"
77.538 +
77.539 + def emit(self, indent):
77.540 + encoding = "# coding: utf-8\n"
77.541 + return encoding + self.defwith + self.suite.emit(indent + INDENT) + self.end
77.542 +
77.543 + def __repr__(self):
77.544 + return "<defwith: %s, %s>" % (self.defwith, self.suite)
77.545 +
77.546 +class TextNode:
77.547 + def __init__(self, value):
77.548 + self.value = value
77.549 +
77.550 + def emit(self, indent, begin_indent=''):
77.551 + return repr(safeunicode(self.value))
77.552 +
77.553 + def __repr__(self):
77.554 + return 't' + repr(self.value)
77.555 +
77.556 +class ExpressionNode:
77.557 + def __init__(self, value, escape=True):
77.558 + self.value = value.strip()
77.559 +
77.560 + # convert ${...} to $(...)
77.561 + if value.startswith('{') and value.endswith('}'):
77.562 + self.value = '(' + self.value[1:-1] + ')'
77.563 +
77.564 + self.escape = escape
77.565 +
77.566 + def emit(self, indent, begin_indent=''):
77.567 + return 'escape_(%s, %s)' % (self.value, bool(self.escape))
77.568 +
77.569 + def __repr__(self):
77.570 + if self.escape:
77.571 + escape = ''
77.572 + else:
77.573 + escape = ':'
77.574 + return "$%s%s" % (escape, self.value)
77.575 +
77.576 +class AssignmentNode:
77.577 + def __init__(self, code):
77.578 + self.code = code
77.579 +
77.580 + def emit(self, indent, begin_indent=''):
77.581 + return indent + self.code + "\n"
77.582 +
77.583 + def __repr__(self):
77.584 + return "<assignment: %s>" % repr(self.code)
77.585 +
77.586 +class LineNode:
77.587 + def __init__(self, nodes):
77.588 + self.nodes = nodes
77.589 +
77.590 + def emit(self, indent, text_indent='', name=''):
77.591 + text = [node.emit('') for node in self.nodes]
77.592 + if text_indent:
77.593 + text = [repr(text_indent)] + text
77.594 +
77.595 + return indent + "extend_([%s])\n" % ", ".join(text)
77.596 +
77.597 + def __repr__(self):
77.598 + return "<line: %s>" % repr(self.nodes)
77.599 +
77.600 +INDENT = ' ' # 4 spaces
77.601 +
77.602 +class BlockNode:
77.603 + def __init__(self, stmt, block, begin_indent=''):
77.604 + self.stmt = stmt
77.605 + self.suite = Parser().read_suite(block)
77.606 + self.begin_indent = begin_indent
77.607 +
77.608 + def emit(self, indent, text_indent=''):
77.609 + text_indent = self.begin_indent + text_indent
77.610 + out = indent + self.stmt + self.suite.emit(indent + INDENT, text_indent)
77.611 + return out
77.612 +
77.613 + def __repr__(self):
77.614 + return "<block: %s, %s>" % (repr(self.stmt), repr(self.suite))
77.615 +
77.616 +class ForNode(BlockNode):
77.617 + def __init__(self, stmt, block, begin_indent=''):
77.618 + self.original_stmt = stmt
77.619 + tok = PythonTokenizer(stmt)
77.620 + tok.consume_till('in')
77.621 + a = stmt[:tok.index] # for i in
77.622 + b = stmt[tok.index:-1] # rest of for stmt excluding :
77.623 + stmt = a + ' loop.setup(' + b.strip() + '):'
77.624 + BlockNode.__init__(self, stmt, block, begin_indent)
77.625 +
77.626 + def __repr__(self):
77.627 + return "<block: %s, %s>" % (repr(self.original_stmt), repr(self.suite))
77.628 +
77.629 +class CodeNode:
77.630 + def __init__(self, stmt, block, begin_indent=''):
77.631 + # compensate one line for $code:
77.632 + self.code = "\n" + block
77.633 +
77.634 + def emit(self, indent, text_indent=''):
77.635 + import re
77.636 + rx = re.compile('^', re.M)
77.637 + return rx.sub(indent, self.code).rstrip(' ')
77.638 +
77.639 + def __repr__(self):
77.640 + return "<code: %s>" % repr(self.code)
77.641 +
77.642 +class StatementNode:
77.643 + def __init__(self, stmt):
77.644 + self.stmt = stmt
77.645 +
77.646 + def emit(self, indent, begin_indent=''):
77.647 + return indent + self.stmt
77.648 +
77.649 + def __repr__(self):
77.650 + return "<stmt: %s>" % repr(self.stmt)
77.651 +
77.652 +class IfNode(BlockNode):
77.653 + pass
77.654 +
77.655 +class ElseNode(BlockNode):
77.656 + pass
77.657 +
77.658 +class ElifNode(BlockNode):
77.659 + pass
77.660 +
77.661 +class DefNode(BlockNode):
77.662 + def __init__(self, *a, **kw):
77.663 + BlockNode.__init__(self, *a, **kw)
77.664 +
77.665 + code = CodeNode("", "")
77.666 + code.code = "self = TemplateResult(); extend_ = self.extend\n"
77.667 + self.suite.sections.insert(0, code)
77.668 +
77.669 + code = CodeNode("", "")
77.670 + code.code = "return self\n"
77.671 + self.suite.sections.append(code)
77.672 +
77.673 + def emit(self, indent, text_indent=''):
77.674 + text_indent = self.begin_indent + text_indent
77.675 + out = indent + self.stmt + self.suite.emit(indent + INDENT, text_indent)
77.676 + return indent + "__lineoffset__ -= 3\n" + out
77.677 +
77.678 +class VarNode:
77.679 + def __init__(self, name, value):
77.680 + self.name = name
77.681 + self.value = value
77.682 +
77.683 + def emit(self, indent, text_indent):
77.684 + return indent + "self[%s] = %s\n" % (repr(self.name), self.value)
77.685 +
77.686 + def __repr__(self):
77.687 + return "<var: %s = %s>" % (self.name, self.value)
77.688 +
77.689 +class SuiteNode:
77.690 + """Suite is a list of sections."""
77.691 + def __init__(self, sections):
77.692 + self.sections = sections
77.693 +
77.694 + def emit(self, indent, text_indent=''):
77.695 + return "\n" + "".join([s.emit(indent, text_indent) for s in self.sections])
77.696 +
77.697 + def __repr__(self):
77.698 + return repr(self.sections)
77.699 +
77.700 +STATEMENT_NODES = {
77.701 + 'for': ForNode,
77.702 + 'while': BlockNode,
77.703 + 'if': IfNode,
77.704 + 'elif': ElifNode,
77.705 + 'else': ElseNode,
77.706 + 'def': DefNode,
77.707 + 'code': CodeNode
77.708 +}
77.709 +
77.710 +KEYWORDS = [
77.711 + "pass",
77.712 + "break",
77.713 + "continue",
77.714 + "return"
77.715 +]
77.716 +
77.717 +TEMPLATE_BUILTIN_NAMES = [
77.718 + "dict", "enumerate", "float", "int", "bool", "list", "long", "reversed",
77.719 + "set", "slice", "tuple", "xrange",
77.720 + "abs", "all", "any", "callable", "chr", "cmp", "divmod", "filter", "hex",
77.721 + "id", "isinstance", "iter", "len", "max", "min", "oct", "ord", "pow", "range",
77.722 + "True", "False",
77.723 + "None",
77.724 + "__import__", # some c-libraries like datetime requires __import__ to present in the namespace
77.725 +]
77.726 +
77.727 +import __builtin__
77.728 +TEMPLATE_BUILTINS = dict([(name, getattr(__builtin__, name)) for name in TEMPLATE_BUILTIN_NAMES if name in __builtin__.__dict__])
77.729 +
77.730 +class ForLoop:
77.731 + """
77.732 + Wrapper for expression in for stament to support loop.xxx helpers.
77.733 +
77.734 + >>> loop = ForLoop()
77.735 + >>> for x in loop.setup(['a', 'b', 'c']):
77.736 + ... print loop.index, loop.revindex, loop.parity, x
77.737 + ...
77.738 + 1 3 odd a
77.739 + 2 2 even b
77.740 + 3 1 odd c
77.741 + >>> loop.index
77.742 + Traceback (most recent call last):
77.743 + ...
77.744 + AttributeError: index
77.745 + """
77.746 + def __init__(self):
77.747 + self._ctx = None
77.748 +
77.749 + def __getattr__(self, name):
77.750 + if self._ctx is None:
77.751 + raise AttributeError, name
77.752 + else:
77.753 + return getattr(self._ctx, name)
77.754 +
77.755 + def setup(self, seq):
77.756 + self._push()
77.757 + return self._ctx.setup(seq)
77.758 +
77.759 + def _push(self):
77.760 + self._ctx = ForLoopContext(self, self._ctx)
77.761 +
77.762 + def _pop(self):
77.763 + self._ctx = self._ctx.parent
77.764 +
77.765 +class ForLoopContext:
77.766 + """Stackable context for ForLoop to support nested for loops.
77.767 + """
77.768 + def __init__(self, forloop, parent):
77.769 + self._forloop = forloop
77.770 + self.parent = parent
77.771 +
77.772 + def setup(self, seq):
77.773 + try:
77.774 + self.length = len(seq)
77.775 + except:
77.776 + self.length = 0
77.777 +
77.778 + self.index = 0
77.779 + for a in seq:
77.780 + self.index += 1
77.781 + yield a
77.782 + self._forloop._pop()
77.783 +
77.784 + index0 = property(lambda self: self.index-1)
77.785 + first = property(lambda self: self.index == 1)
77.786 + last = property(lambda self: self.index == self.length)
77.787 + odd = property(lambda self: self.index % 2 == 1)
77.788 + even = property(lambda self: self.index % 2 == 0)
77.789 + parity = property(lambda self: ['odd', 'even'][self.even])
77.790 + revindex0 = property(lambda self: self.length - self.index)
77.791 + revindex = property(lambda self: self.length - self.index + 1)
77.792 +
77.793 +class BaseTemplate:
77.794 + def __init__(self, code, filename, filter, globals, builtins):
77.795 + self.filename = filename
77.796 + self.filter = filter
77.797 + self._globals = globals
77.798 + self._builtins = builtins
77.799 + if code:
77.800 + self.t = self._compile(code)
77.801 + else:
77.802 + self.t = lambda: ''
77.803 +
77.804 + def _compile(self, code):
77.805 + env = self.make_env(self._globals or {}, self._builtins)
77.806 + exec(code, env)
77.807 + return env['__template__']
77.808 +
77.809 + def __call__(self, *a, **kw):
77.810 + __hidetraceback__ = True
77.811 + return self.t(*a, **kw)
77.812 +
77.813 + def make_env(self, globals, builtins):
77.814 + return dict(globals,
77.815 + __builtins__=builtins,
77.816 + ForLoop=ForLoop,
77.817 + TemplateResult=TemplateResult,
77.818 + escape_=self._escape,
77.819 + join_=self._join
77.820 + )
77.821 + def _join(self, *items):
77.822 + return u"".join(items)
77.823 +
77.824 + def _escape(self, value, escape=False):
77.825 + if value is None:
77.826 + value = ''
77.827 +
77.828 + value = safeunicode(value)
77.829 + if escape and self.filter:
77.830 + value = self.filter(value)
77.831 + return value
77.832 +
77.833 +class Template(BaseTemplate):
77.834 + CONTENT_TYPES = {
77.835 + '.html' : 'text/html; charset=utf-8',
77.836 + '.xhtml' : 'application/xhtml+xml; charset=utf-8',
77.837 + '.txt' : 'text/plain',
77.838 + }
77.839 + FILTERS = {
77.840 + '.html': websafe,
77.841 + '.xhtml': websafe,
77.842 + '.xml': websafe
77.843 + }
77.844 + globals = {}
77.845 +
77.846 + def __init__(self, text, filename='<template>', filter=None, globals=None, builtins=None, extensions=None):
77.847 + self.extensions = extensions or []
77.848 + text = Template.normalize_text(text)
77.849 + code = self.compile_template(text, filename)
77.850 +
77.851 + _, ext = os.path.splitext(filename)
77.852 + filter = filter or self.FILTERS.get(ext, None)
77.853 + self.content_type = self.CONTENT_TYPES.get(ext, None)
77.854 +
77.855 + if globals is None:
77.856 + globals = self.globals
77.857 + if builtins is None:
77.858 + builtins = TEMPLATE_BUILTINS
77.859 +
77.860 + BaseTemplate.__init__(self, code=code, filename=filename, filter=filter, globals=globals, builtins=builtins)
77.861 +
77.862 + def normalize_text(text):
77.863 + """Normalizes template text by correcting \r\n, tabs and BOM chars."""
77.864 + text = text.replace('\r\n', '\n').replace('\r', '\n').expandtabs()
77.865 + if not text.endswith('\n'):
77.866 + text += '\n'
77.867 +
77.868 + # ignore BOM chars at the begining of template
77.869 + BOM = '\xef\xbb\xbf'
77.870 + if isinstance(text, str) and text.startswith(BOM):
77.871 + text = text[len(BOM):]
77.872 +
77.873 + # support fort \$ for backward-compatibility
77.874 + text = text.replace(r'\$', '$$')
77.875 + return text
77.876 + normalize_text = staticmethod(normalize_text)
77.877 +
77.878 + def __call__(self, *a, **kw):
77.879 + __hidetraceback__ = True
77.880 + import webapi as web
77.881 + if 'headers' in web.ctx and self.content_type:
77.882 + web.header('Content-Type', self.content_type, unique=True)
77.883 +
77.884 + return BaseTemplate.__call__(self, *a, **kw)
77.885 +
77.886 + def generate_code(text, filename, parser=None):
77.887 + # parse the text
77.888 + parser = parser or Parser()
77.889 + rootnode = parser.parse(text, filename)
77.890 +
77.891 + # generate python code from the parse tree
77.892 + code = rootnode.emit(indent="").strip()
77.893 + return safestr(code)
77.894 +
77.895 + generate_code = staticmethod(generate_code)
77.896 +
77.897 + def create_parser(self):
77.898 + p = Parser()
77.899 + for ext in self.extensions:
77.900 + p = ext(p)
77.901 + return p
77.902 +
77.903 + def compile_template(self, template_string, filename):
77.904 + code = Template.generate_code(template_string, filename, parser=self.create_parser())
77.905 +
77.906 + def get_source_line(filename, lineno):
77.907 + try:
77.908 + lines = open(filename).read().splitlines()
77.909 + return lines[lineno]
77.910 + except:
77.911 + return None
77.912 +
77.913 + try:
77.914 + # compile the code first to report the errors, if any, with the filename
77.915 + compiled_code = compile(code, filename, 'exec')
77.916 + except SyntaxError, e:
77.917 + # display template line that caused the error along with the traceback.
77.918 + try:
77.919 + e.msg += '\n\nTemplate traceback:\n File %s, line %s\n %s' % \
77.920 + (repr(e.filename), e.lineno, get_source_line(e.filename, e.lineno-1))
77.921 + except:
77.922 + pass
77.923 + raise
77.924 +
77.925 + # make sure code is safe - but not with jython, it doesn't have a working compiler module
77.926 + if not sys.platform.startswith('java'):
77.927 + try:
77.928 + import compiler
77.929 + ast = compiler.parse(code)
77.930 + SafeVisitor().walk(ast, filename)
77.931 + except ImportError:
77.932 + warnings.warn("Unabled to import compiler module. Unable to check templates for safety.")
77.933 + else:
77.934 + warnings.warn("SECURITY ISSUE: You are using Jython, which does not support checking templates for safety. Your templates can execute arbitrary code.")
77.935 +
77.936 + return compiled_code
77.937 +
77.938 +class CompiledTemplate(Template):
77.939 + def __init__(self, f, filename):
77.940 + Template.__init__(self, '', filename)
77.941 + self.t = f
77.942 +
77.943 + def compile_template(self, *a):
77.944 + return None
77.945 +
77.946 + def _compile(self, *a):
77.947 + return None
77.948 +
77.949 +class Render:
77.950 + """The most preferred way of using templates.
77.951 +
77.952 + render = web.template.render('templates')
77.953 + print render.foo()
77.954 +
77.955 + Optional parameter can be `base` can be used to pass output of
77.956 + every template through the base template.
77.957 +
77.958 + render = web.template.render('templates', base='layout')
77.959 + """
77.960 + def __init__(self, loc='templates', cache=None, base=None, **keywords):
77.961 + self._loc = loc
77.962 + self._keywords = keywords
77.963 +
77.964 + if cache is None:
77.965 + cache = not config.get('debug', False)
77.966 +
77.967 + if cache:
77.968 + self._cache = {}
77.969 + else:
77.970 + self._cache = None
77.971 +
77.972 + if base and not hasattr(base, '__call__'):
77.973 + # make base a function, so that it can be passed to sub-renders
77.974 + self._base = lambda page: self._template(base)(page)
77.975 + else:
77.976 + self._base = base
77.977 +
77.978 + def _add_global(self, obj, name=None):
77.979 + """Add a global to this rendering instance."""
77.980 + if 'globals' not in self._keywords: self._keywords['globals'] = {}
77.981 + if not name:
77.982 + name = obj.__name__
77.983 + self._keywords['globals'][name] = obj
77.984 +
77.985 + def _lookup(self, name):
77.986 + path = os.path.join(self._loc, name)
77.987 + if os.path.isdir(path):
77.988 + return 'dir', path
77.989 + else:
77.990 + path = self._findfile(path)
77.991 + if path:
77.992 + return 'file', path
77.993 + else:
77.994 + return 'none', None
77.995 +
77.996 + def _load_template(self, name):
77.997 + kind, path = self._lookup(name)
77.998 +
77.999 + if kind == 'dir':
77.1000 + return Render(path, cache=self._cache is not None, base=self._base, **self._keywords)
77.1001 + elif kind == 'file':
77.1002 + return Template(open(path).read(), filename=path, **self._keywords)
77.1003 + else:
77.1004 + raise AttributeError, "No template named " + name
77.1005 +
77.1006 + def _findfile(self, path_prefix):
77.1007 + p = [f for f in glob.glob(path_prefix + '.*') if not f.endswith('~')] # skip backup files
77.1008 + p.sort() # sort the matches for deterministic order
77.1009 + return p and p[0]
77.1010 +
77.1011 + def _template(self, name):
77.1012 + if self._cache is not None:
77.1013 + if name not in self._cache:
77.1014 + self._cache[name] = self._load_template(name)
77.1015 + return self._cache[name]
77.1016 + else:
77.1017 + return self._load_template(name)
77.1018 +
77.1019 + def __getattr__(self, name):
77.1020 + t = self._template(name)
77.1021 + if self._base and isinstance(t, Template):
77.1022 + def template(*a, **kw):
77.1023 + return self._base(t(*a, **kw))
77.1024 + return template
77.1025 + else:
77.1026 + return self._template(name)
77.1027 +
77.1028 +class GAE_Render(Render):
77.1029 + # Render gets over-written. make a copy here.
77.1030 + super = Render
77.1031 + def __init__(self, loc, *a, **kw):
77.1032 + GAE_Render.super.__init__(self, loc, *a, **kw)
77.1033 +
77.1034 + import types
77.1035 + if isinstance(loc, types.ModuleType):
77.1036 + self.mod = loc
77.1037 + else:
77.1038 + name = loc.rstrip('/').replace('/', '.')
77.1039 + self.mod = __import__(name, None, None, ['x'])
77.1040 +
77.1041 + self.mod.__dict__.update(kw.get('builtins', TEMPLATE_BUILTINS))
77.1042 + self.mod.__dict__.update(Template.globals)
77.1043 + self.mod.__dict__.update(kw.get('globals', {}))
77.1044 +
77.1045 + def _load_template(self, name):
77.1046 + t = getattr(self.mod, name)
77.1047 + import types
77.1048 + if isinstance(t, types.ModuleType):
77.1049 + return GAE_Render(t, cache=self._cache is not None, base=self._base, **self._keywords)
77.1050 + else:
77.1051 + return t
77.1052 +
77.1053 +render = Render
77.1054 +# setup render for Google App Engine.
77.1055 +try:
77.1056 + from google import appengine
77.1057 + render = Render = GAE_Render
77.1058 +except ImportError:
77.1059 + pass
77.1060 +
77.1061 +def frender(path, **keywords):
77.1062 + """Creates a template from the given file path.
77.1063 + """
77.1064 + return Template(open(path).read(), filename=path, **keywords)
77.1065 +
77.1066 +def compile_templates(root):
77.1067 + """Compiles templates to python code."""
77.1068 + re_start = re_compile('^', re.M)
77.1069 +
77.1070 + for dirpath, dirnames, filenames in os.walk(root):
77.1071 + filenames = [f for f in filenames if not f.startswith('.') and not f.endswith('~') and not f.startswith('__init__.py')]
77.1072 +
77.1073 + for d in dirnames[:]:
77.1074 + if d.startswith('.'):
77.1075 + dirnames.remove(d) # don't visit this dir
77.1076 +
77.1077 + out = open(os.path.join(dirpath, '__init__.py'), 'w')
77.1078 + out.write('from web.template import CompiledTemplate, ForLoop, TemplateResult\n\n')
77.1079 + if dirnames:
77.1080 + out.write("import " + ", ".join(dirnames))
77.1081 + out.write("\n")
77.1082 +
77.1083 + for f in filenames:
77.1084 + path = os.path.join(dirpath, f)
77.1085 +
77.1086 + if '.' in f:
77.1087 + name, _ = f.split('.', 1)
77.1088 + else:
77.1089 + name = f
77.1090 +
77.1091 + text = open(path).read()
77.1092 + text = Template.normalize_text(text)
77.1093 + code = Template.generate_code(text, path)
77.1094 +
77.1095 + code = code.replace("__template__", name, 1)
77.1096 +
77.1097 + out.write(code)
77.1098 +
77.1099 + out.write('\n\n')
77.1100 + out.write('%s = CompiledTemplate(%s, %s)\n' % (name, name, repr(path)))
77.1101 + out.write("join_ = %s._join; escape_ = %s._escape\n\n" % (name, name))
77.1102 +
77.1103 + # create template to make sure it compiles
77.1104 + t = Template(open(path).read(), path)
77.1105 + out.close()
77.1106 +
77.1107 +class ParseError(Exception):
77.1108 + pass
77.1109 +
77.1110 +class SecurityError(Exception):
77.1111 + """The template seems to be trying to do something naughty."""
77.1112 + pass
77.1113 +
77.1114 +# Enumerate all the allowed AST nodes
77.1115 +ALLOWED_AST_NODES = [
77.1116 + "Add", "And",
77.1117 +# "AssAttr",
77.1118 + "AssList", "AssName", "AssTuple",
77.1119 +# "Assert",
77.1120 + "Assign", "AugAssign",
77.1121 +# "Backquote",
77.1122 + "Bitand", "Bitor", "Bitxor", "Break",
77.1123 + "CallFunc","Class", "Compare", "Const", "Continue",
77.1124 + "Decorators", "Dict", "Discard", "Div",
77.1125 + "Ellipsis", "EmptyNode",
77.1126 +# "Exec",
77.1127 + "Expression", "FloorDiv", "For",
77.1128 +# "From",
77.1129 + "Function",
77.1130 + "GenExpr", "GenExprFor", "GenExprIf", "GenExprInner",
77.1131 + "Getattr",
77.1132 +# "Global",
77.1133 + "If", "IfExp",
77.1134 +# "Import",
77.1135 + "Invert", "Keyword", "Lambda", "LeftShift",
77.1136 + "List", "ListComp", "ListCompFor", "ListCompIf", "Mod",
77.1137 + "Module",
77.1138 + "Mul", "Name", "Not", "Or", "Pass", "Power",
77.1139 +# "Print", "Printnl", "Raise",
77.1140 + "Return", "RightShift", "Slice", "Sliceobj",
77.1141 + "Stmt", "Sub", "Subscript",
77.1142 +# "TryExcept", "TryFinally",
77.1143 + "Tuple", "UnaryAdd", "UnarySub",
77.1144 + "While", "With", "Yield",
77.1145 +]
77.1146 +
77.1147 +class SafeVisitor(object):
77.1148 + """
77.1149 + Make sure code is safe by walking through the AST.
77.1150 +
77.1151 + Code considered unsafe if:
77.1152 + * it has restricted AST nodes
77.1153 + * it is trying to access resricted attributes
77.1154 +
77.1155 + Adopted from http://www.zafar.se/bkz/uploads/safe.txt (public domain, Babar K. Zafar)
77.1156 + """
77.1157 + def __init__(self):
77.1158 + "Initialize visitor by generating callbacks for all AST node types."
77.1159 + self.errors = []
77.1160 +
77.1161 + def walk(self, ast, filename):
77.1162 + "Validate each node in AST and raise SecurityError if the code is not safe."
77.1163 + self.filename = filename
77.1164 + self.visit(ast)
77.1165 +
77.1166 + if self.errors:
77.1167 + raise SecurityError, '\n'.join([str(err) for err in self.errors])
77.1168 +
77.1169 + def visit(self, node, *args):
77.1170 + "Recursively validate node and all of its children."
77.1171 + def classname(obj):
77.1172 + return obj.__class__.__name__
77.1173 + nodename = classname(node)
77.1174 + fn = getattr(self, 'visit' + nodename, None)
77.1175 +
77.1176 + if fn:
77.1177 + fn(node, *args)
77.1178 + else:
77.1179 + if nodename not in ALLOWED_AST_NODES:
77.1180 + self.fail(node, *args)
77.1181 +
77.1182 + for child in node.getChildNodes():
77.1183 + self.visit(child, *args)
77.1184 +
77.1185 + def visitName(self, node, *args):
77.1186 + "Disallow any attempts to access a restricted attr."
77.1187 + #self.assert_attr(node.getChildren()[0], node)
77.1188 + pass
77.1189 +
77.1190 + def visitGetattr(self, node, *args):
77.1191 + "Disallow any attempts to access a restricted attribute."
77.1192 + self.assert_attr(node.attrname, node)
77.1193 +
77.1194 + def assert_attr(self, attrname, node):
77.1195 + if self.is_unallowed_attr(attrname):
77.1196 + lineno = self.get_node_lineno(node)
77.1197 + e = SecurityError("%s:%d - access to attribute '%s' is denied" % (self.filename, lineno, attrname))
77.1198 + self.errors.append(e)
77.1199 +
77.1200 + def is_unallowed_attr(self, name):
77.1201 + return name.startswith('_') \
77.1202 + or name.startswith('func_') \
77.1203 + or name.startswith('im_')
77.1204 +
77.1205 + def get_node_lineno(self, node):
77.1206 + return (node.lineno) and node.lineno or 0
77.1207 +
77.1208 + def fail(self, node, *args):
77.1209 + "Default callback for unallowed AST nodes."
77.1210 + lineno = self.get_node_lineno(node)
77.1211 + nodename = node.__class__.__name__
77.1212 + e = SecurityError("%s:%d - execution of '%s' statements is denied" % (self.filename, lineno, nodename))
77.1213 + self.errors.append(e)
77.1214 +
77.1215 +class TemplateResult(object, DictMixin):
77.1216 + """Dictionary like object for storing template output.
77.1217 +
77.1218 + The result of a template execution is usally a string, but sometimes it
77.1219 + contains attributes set using $var. This class provides a simple
77.1220 + dictionary like interface for storing the output of the template and the
77.1221 + attributes. The output is stored with a special key __body__. Convering
77.1222 + the the TemplateResult to string or unicode returns the value of __body__.
77.1223 +
77.1224 + When the template is in execution, the output is generated part by part
77.1225 + and those parts are combined at the end. Parts are added to the
77.1226 + TemplateResult by calling the `extend` method and the parts are combined
77.1227 + seemlessly when __body__ is accessed.
77.1228 +
77.1229 + >>> d = TemplateResult(__body__='hello, world', x='foo')
77.1230 + >>> d
77.1231 + <TemplateResult: {'__body__': 'hello, world', 'x': 'foo'}>
77.1232 + >>> print d
77.1233 + hello, world
77.1234 + >>> d.x
77.1235 + 'foo'
77.1236 + >>> d = TemplateResult()
77.1237 + >>> d.extend([u'hello', u'world'])
77.1238 + >>> d
77.1239 + <TemplateResult: {'__body__': u'helloworld'}>
77.1240 + """
77.1241 + def __init__(self, *a, **kw):
77.1242 + self.__dict__["_d"] = dict(*a, **kw)
77.1243 + self._d.setdefault("__body__", u'')
77.1244 +
77.1245 + self.__dict__['_parts'] = []
77.1246 + self.__dict__["extend"] = self._parts.extend
77.1247 +
77.1248 + self._d.setdefault("__body__", None)
77.1249 +
77.1250 + def keys(self):
77.1251 + return self._d.keys()
77.1252 +
77.1253 + def _prepare_body(self):
77.1254 + """Prepare value of __body__ by joining parts.
77.1255 + """
77.1256 + if self._parts:
77.1257 + value = u"".join(self._parts)
77.1258 + self._parts[:] = []
77.1259 + body = self._d.get('__body__')
77.1260 + if body:
77.1261 + self._d['__body__'] = body + value
77.1262 + else:
77.1263 + self._d['__body__'] = value
77.1264 +
77.1265 + def __getitem__(self, name):
77.1266 + if name == "__body__":
77.1267 + self._prepare_body()
77.1268 + return self._d[name]
77.1269 +
77.1270 + def __setitem__(self, name, value):
77.1271 + if name == "__body__":
77.1272 + self._prepare_body()
77.1273 + return self._d.__setitem__(name, value)
77.1274 +
77.1275 + def __delitem__(self, name):
77.1276 + if name == "__body__":
77.1277 + self._prepare_body()
77.1278 + return self._d.__delitem__(name)
77.1279 +
77.1280 + def __getattr__(self, key):
77.1281 + try:
77.1282 + return self[key]
77.1283 + except KeyError, k:
77.1284 + raise AttributeError, k
77.1285 +
77.1286 + def __setattr__(self, key, value):
77.1287 + self[key] = value
77.1288 +
77.1289 + def __delattr__(self, key):
77.1290 + try:
77.1291 + del self[key]
77.1292 + except KeyError, k:
77.1293 + raise AttributeError, k
77.1294 +
77.1295 + def __unicode__(self):
77.1296 + self._prepare_body()
77.1297 + return self["__body__"]
77.1298 +
77.1299 + def __str__(self):
77.1300 + self._prepare_body()
77.1301 + return self["__body__"].encode('utf-8')
77.1302 +
77.1303 + def __repr__(self):
77.1304 + self._prepare_body()
77.1305 + return "<TemplateResult: %s>" % self._d
77.1306 +
77.1307 +def test():
77.1308 + r"""Doctest for testing template module.
77.1309 +
77.1310 + Define a utility function to run template test.
77.1311 +
77.1312 + >>> class TestResult:
77.1313 + ... def __init__(self, t): self.t = t
77.1314 + ... def __getattr__(self, name): return getattr(self.t, name)
77.1315 + ... def __repr__(self): return repr(unicode(self))
77.1316 + ...
77.1317 + >>> def t(code, **keywords):
77.1318 + ... tmpl = Template(code, **keywords)
77.1319 + ... return lambda *a, **kw: TestResult(tmpl(*a, **kw))
77.1320 + ...
77.1321 +
77.1322 + Simple tests.
77.1323 +
77.1324 + >>> t('1')()
77.1325 + u'1\n'
77.1326 + >>> t('$def with ()\n1')()
77.1327 + u'1\n'
77.1328 + >>> t('$def with (a)\n$a')(1)
77.1329 + u'1\n'
77.1330 + >>> t('$def with (a=0)\n$a')(1)
77.1331 + u'1\n'
77.1332 + >>> t('$def with (a=0)\n$a')(a=1)
77.1333 + u'1\n'
77.1334 +
77.1335 + Test complicated expressions.
77.1336 +
77.1337 + >>> t('$def with (x)\n$x.upper()')('hello')
77.1338 + u'HELLO\n'
77.1339 + >>> t('$(2 * 3 + 4 * 5)')()
77.1340 + u'26\n'
77.1341 + >>> t('${2 * 3 + 4 * 5}')()
77.1342 + u'26\n'
77.1343 + >>> t('$def with (limit)\nkeep $(limit)ing.')('go')
77.1344 + u'keep going.\n'
77.1345 + >>> t('$def with (a)\n$a.b[0]')(storage(b=[1]))
77.1346 + u'1\n'
77.1347 +
77.1348 + Test html escaping.
77.1349 +
77.1350 + >>> t('$def with (x)\n$x', filename='a.html')('<html>')
77.1351 + u'<html>\n'
77.1352 + >>> t('$def with (x)\n$x', filename='a.txt')('<html>')
77.1353 + u'<html>\n'
77.1354 +
77.1355 + Test if, for and while.
77.1356 +
77.1357 + >>> t('$if 1: 1')()
77.1358 + u'1\n'
77.1359 + >>> t('$if 1:\n 1')()
77.1360 + u'1\n'
77.1361 + >>> t('$if 1:\n 1\\')()
77.1362 + u'1'
77.1363 + >>> t('$if 0: 0\n$elif 1: 1')()
77.1364 + u'1\n'
77.1365 + >>> t('$if 0: 0\n$elif None: 0\n$else: 1')()
77.1366 + u'1\n'
77.1367 + >>> t('$if 0 < 1 and 1 < 2: 1')()
77.1368 + u'1\n'
77.1369 + >>> t('$for x in [1, 2, 3]: $x')()
77.1370 + u'1\n2\n3\n'
77.1371 + >>> t('$def with (d)\n$for k, v in d.iteritems(): $k')({1: 1})
77.1372 + u'1\n'
77.1373 + >>> t('$for x in [1, 2, 3]:\n\t$x')()
77.1374 + u' 1\n 2\n 3\n'
77.1375 + >>> t('$def with (a)\n$while a and a.pop():1')([1, 2, 3])
77.1376 + u'1\n1\n1\n'
77.1377 +
77.1378 + The space after : must be ignored.
77.1379 +
77.1380 + >>> t('$if True: foo')()
77.1381 + u'foo\n'
77.1382 +
77.1383 + Test loop.xxx.
77.1384 +
77.1385 + >>> t("$for i in range(5):$loop.index, $loop.parity")()
77.1386 + u'1, odd\n2, even\n3, odd\n4, even\n5, odd\n'
77.1387 + >>> t("$for i in range(2):\n $for j in range(2):$loop.parent.parity $loop.parity")()
77.1388 + u'odd odd\nodd even\neven odd\neven even\n'
77.1389 +
77.1390 + Test assignment.
77.1391 +
77.1392 + >>> t('$ a = 1\n$a')()
77.1393 + u'1\n'
77.1394 + >>> t('$ a = [1]\n$a[0]')()
77.1395 + u'1\n'
77.1396 + >>> t('$ a = {1: 1}\n$a.keys()[0]')()
77.1397 + u'1\n'
77.1398 + >>> t('$ a = []\n$if not a: 1')()
77.1399 + u'1\n'
77.1400 + >>> t('$ a = {}\n$if not a: 1')()
77.1401 + u'1\n'
77.1402 + >>> t('$ a = -1\n$a')()
77.1403 + u'-1\n'
77.1404 + >>> t('$ a = "1"\n$a')()
77.1405 + u'1\n'
77.1406 +
77.1407 + Test comments.
77.1408 +
77.1409 + >>> t('$# 0')()
77.1410 + u'\n'
77.1411 + >>> t('hello$#comment1\nhello$#comment2')()
77.1412 + u'hello\nhello\n'
77.1413 + >>> t('$#comment0\nhello$#comment1\nhello$#comment2')()
77.1414 + u'\nhello\nhello\n'
77.1415 +
77.1416 + Test unicode.
77.1417 +
77.1418 + >>> t('$def with (a)\n$a')(u'\u203d')
77.1419 + u'\u203d\n'
77.1420 + >>> t('$def with (a)\n$a')(u'\u203d'.encode('utf-8'))
77.1421 + u'\u203d\n'
77.1422 + >>> t(u'$def with (a)\n$a $:a')(u'\u203d')
77.1423 + u'\u203d \u203d\n'
77.1424 + >>> t(u'$def with ()\nfoo')()
77.1425 + u'foo\n'
77.1426 + >>> def f(x): return x
77.1427 + ...
77.1428 + >>> t(u'$def with (f)\n$:f("x")')(f)
77.1429 + u'x\n'
77.1430 + >>> t('$def with (f)\n$:f("x")')(f)
77.1431 + u'x\n'
77.1432 +
77.1433 + Test dollar escaping.
77.1434 +
77.1435 + >>> t("Stop, $$money isn't evaluated.")()
77.1436 + u"Stop, $money isn't evaluated.\n"
77.1437 + >>> t("Stop, \$money isn't evaluated.")()
77.1438 + u"Stop, $money isn't evaluated.\n"
77.1439 +
77.1440 + Test space sensitivity.
77.1441 +
77.1442 + >>> t('$def with (x)\n$x')(1)
77.1443 + u'1\n'
77.1444 + >>> t('$def with(x ,y)\n$x')(1, 1)
77.1445 + u'1\n'
77.1446 + >>> t('$(1 + 2*3 + 4)')()
77.1447 + u'11\n'
77.1448 +
77.1449 + Make sure globals are working.
77.1450 +
77.1451 + >>> t('$x')()
77.1452 + Traceback (most recent call last):
77.1453 + ...
77.1454 + NameError: global name 'x' is not defined
77.1455 + >>> t('$x', globals={'x': 1})()
77.1456 + u'1\n'
77.1457 +
77.1458 + Can't change globals.
77.1459 +
77.1460 + >>> t('$ x = 2\n$x', globals={'x': 1})()
77.1461 + u'2\n'
77.1462 + >>> t('$ x = x + 1\n$x', globals={'x': 1})()
77.1463 + Traceback (most recent call last):
77.1464 + ...
77.1465 + UnboundLocalError: local variable 'x' referenced before assignment
77.1466 +
77.1467 + Make sure builtins are customizable.
77.1468 +
77.1469 + >>> t('$min(1, 2)')()
77.1470 + u'1\n'
77.1471 + >>> t('$min(1, 2)', builtins={})()
77.1472 + Traceback (most recent call last):
77.1473 + ...
77.1474 + NameError: global name 'min' is not defined
77.1475 +
77.1476 + Test vars.
77.1477 +
77.1478 + >>> x = t('$var x: 1')()
77.1479 + >>> x.x
77.1480 + u'1'
77.1481 + >>> x = t('$var x = 1')()
77.1482 + >>> x.x
77.1483 + 1
77.1484 + >>> x = t('$var x: \n foo\n bar')()
77.1485 + >>> x.x
77.1486 + u'foo\nbar\n'
77.1487 +
77.1488 + Test BOM chars.
77.1489 +
77.1490 + >>> t('\xef\xbb\xbf$def with(x)\n$x')('foo')
77.1491 + u'foo\n'
77.1492 +
77.1493 + Test for with weird cases.
77.1494 +
77.1495 + >>> t('$for i in range(10)[1:5]:\n $i')()
77.1496 + u'1\n2\n3\n4\n'
77.1497 + >>> t("$for k, v in {'a': 1, 'b': 2}.items():\n $k $v")()
77.1498 + u'a 1\nb 2\n'
77.1499 + >>> t("$for k, v in ({'a': 1, 'b': 2}.items():\n $k $v")()
77.1500 + Traceback (most recent call last):
77.1501 + ...
77.1502 + SyntaxError: invalid syntax
77.1503 +
77.1504 + Test datetime.
77.1505 +
77.1506 + >>> import datetime
77.1507 + >>> t("$def with (date)\n$date.strftime('%m %Y')")(datetime.datetime(2009, 1, 1))
77.1508 + u'01 2009\n'
77.1509 + """
77.1510 + pass
77.1511 +
77.1512 +if __name__ == "__main__":
77.1513 + import sys
77.1514 + if '--compile' in sys.argv:
77.1515 + compile_templates(sys.argv[2])
77.1516 + else:
77.1517 + import doctest
77.1518 + doctest.testmod()
78.1 Binary file OpenSecurity/install/web.py-0.37/web/template.pyc has changed
79.1 --- /dev/null Thu Jan 01 00:00:00 1970 +0000
79.2 +++ b/OpenSecurity/install/web.py-0.37/web/test.py Mon Dec 02 14:02:05 2013 +0100
79.3 @@ -0,0 +1,51 @@
79.4 +"""test utilities
79.5 +(part of web.py)
79.6 +"""
79.7 +import unittest
79.8 +import sys, os
79.9 +import web
79.10 +
79.11 +TestCase = unittest.TestCase
79.12 +TestSuite = unittest.TestSuite
79.13 +
79.14 +def load_modules(names):
79.15 + return [__import__(name, None, None, "x") for name in names]
79.16 +
79.17 +def module_suite(module, classnames=None):
79.18 + """Makes a suite from a module."""
79.19 + if classnames:
79.20 + return unittest.TestLoader().loadTestsFromNames(classnames, module)
79.21 + elif hasattr(module, 'suite'):
79.22 + return module.suite()
79.23 + else:
79.24 + return unittest.TestLoader().loadTestsFromModule(module)
79.25 +
79.26 +def doctest_suite(module_names):
79.27 + """Makes a test suite from doctests."""
79.28 + import doctest
79.29 + suite = TestSuite()
79.30 + for mod in load_modules(module_names):
79.31 + suite.addTest(doctest.DocTestSuite(mod))
79.32 + return suite
79.33 +
79.34 +def suite(module_names):
79.35 + """Creates a suite from multiple modules."""
79.36 + suite = TestSuite()
79.37 + for mod in load_modules(module_names):
79.38 + suite.addTest(module_suite(mod))
79.39 + return suite
79.40 +
79.41 +def runTests(suite):
79.42 + runner = unittest.TextTestRunner()
79.43 + return runner.run(suite)
79.44 +
79.45 +def main(suite=None):
79.46 + if not suite:
79.47 + main_module = __import__('__main__')
79.48 + # allow command line switches
79.49 + args = [a for a in sys.argv[1:] if not a.startswith('-')]
79.50 + suite = module_suite(main_module, args or None)
79.51 +
79.52 + result = runTests(suite)
79.53 + sys.exit(not result.wasSuccessful())
79.54 +
80.1 --- /dev/null Thu Jan 01 00:00:00 1970 +0000
80.2 +++ b/OpenSecurity/install/web.py-0.37/web/utils.py Mon Dec 02 14:02:05 2013 +0100
80.3 @@ -0,0 +1,1526 @@
80.4 +#!/usr/bin/env python
80.5 +"""
80.6 +General Utilities
80.7 +(part of web.py)
80.8 +"""
80.9 +
80.10 +__all__ = [
80.11 + "Storage", "storage", "storify",
80.12 + "Counter", "counter",
80.13 + "iters",
80.14 + "rstrips", "lstrips", "strips",
80.15 + "safeunicode", "safestr", "utf8",
80.16 + "TimeoutError", "timelimit",
80.17 + "Memoize", "memoize",
80.18 + "re_compile", "re_subm",
80.19 + "group", "uniq", "iterview",
80.20 + "IterBetter", "iterbetter",
80.21 + "safeiter", "safewrite",
80.22 + "dictreverse", "dictfind", "dictfindall", "dictincr", "dictadd",
80.23 + "requeue", "restack",
80.24 + "listget", "intget", "datestr",
80.25 + "numify", "denumify", "commify", "dateify",
80.26 + "nthstr", "cond",
80.27 + "CaptureStdout", "capturestdout", "Profile", "profile",
80.28 + "tryall",
80.29 + "ThreadedDict", "threadeddict",
80.30 + "autoassign",
80.31 + "to36",
80.32 + "safemarkdown",
80.33 + "sendmail"
80.34 +]
80.35 +
80.36 +import re, sys, time, threading, itertools, traceback, os
80.37 +
80.38 +try:
80.39 + import subprocess
80.40 +except ImportError:
80.41 + subprocess = None
80.42 +
80.43 +try: import datetime
80.44 +except ImportError: pass
80.45 +
80.46 +try: set
80.47 +except NameError:
80.48 + from sets import Set as set
80.49 +
80.50 +try:
80.51 + from threading import local as threadlocal
80.52 +except ImportError:
80.53 + from python23 import threadlocal
80.54 +
80.55 +class Storage(dict):
80.56 + """
80.57 + A Storage object is like a dictionary except `obj.foo` can be used
80.58 + in addition to `obj['foo']`.
80.59 +
80.60 + >>> o = storage(a=1)
80.61 + >>> o.a
80.62 + 1
80.63 + >>> o['a']
80.64 + 1
80.65 + >>> o.a = 2
80.66 + >>> o['a']
80.67 + 2
80.68 + >>> del o.a
80.69 + >>> o.a
80.70 + Traceback (most recent call last):
80.71 + ...
80.72 + AttributeError: 'a'
80.73 +
80.74 + """
80.75 + def __getattr__(self, key):
80.76 + try:
80.77 + return self[key]
80.78 + except KeyError, k:
80.79 + raise AttributeError, k
80.80 +
80.81 + def __setattr__(self, key, value):
80.82 + self[key] = value
80.83 +
80.84 + def __delattr__(self, key):
80.85 + try:
80.86 + del self[key]
80.87 + except KeyError, k:
80.88 + raise AttributeError, k
80.89 +
80.90 + def __repr__(self):
80.91 + return '<Storage ' + dict.__repr__(self) + '>'
80.92 +
80.93 +storage = Storage
80.94 +
80.95 +def storify(mapping, *requireds, **defaults):
80.96 + """
80.97 + Creates a `storage` object from dictionary `mapping`, raising `KeyError` if
80.98 + d doesn't have all of the keys in `requireds` and using the default
80.99 + values for keys found in `defaults`.
80.100 +
80.101 + For example, `storify({'a':1, 'c':3}, b=2, c=0)` will return the equivalent of
80.102 + `storage({'a':1, 'b':2, 'c':3})`.
80.103 +
80.104 + If a `storify` value is a list (e.g. multiple values in a form submission),
80.105 + `storify` returns the last element of the list, unless the key appears in
80.106 + `defaults` as a list. Thus:
80.107 +
80.108 + >>> storify({'a':[1, 2]}).a
80.109 + 2
80.110 + >>> storify({'a':[1, 2]}, a=[]).a
80.111 + [1, 2]
80.112 + >>> storify({'a':1}, a=[]).a
80.113 + [1]
80.114 + >>> storify({}, a=[]).a
80.115 + []
80.116 +
80.117 + Similarly, if the value has a `value` attribute, `storify will return _its_
80.118 + value, unless the key appears in `defaults` as a dictionary.
80.119 +
80.120 + >>> storify({'a':storage(value=1)}).a
80.121 + 1
80.122 + >>> storify({'a':storage(value=1)}, a={}).a
80.123 + <Storage {'value': 1}>
80.124 + >>> storify({}, a={}).a
80.125 + {}
80.126 +
80.127 + Optionally, keyword parameter `_unicode` can be passed to convert all values to unicode.
80.128 +
80.129 + >>> storify({'x': 'a'}, _unicode=True)
80.130 + <Storage {'x': u'a'}>
80.131 + >>> storify({'x': storage(value='a')}, x={}, _unicode=True)
80.132 + <Storage {'x': <Storage {'value': 'a'}>}>
80.133 + >>> storify({'x': storage(value='a')}, _unicode=True)
80.134 + <Storage {'x': u'a'}>
80.135 + """
80.136 + _unicode = defaults.pop('_unicode', False)
80.137 +
80.138 + # if _unicode is callable object, use it convert a string to unicode.
80.139 + to_unicode = safeunicode
80.140 + if _unicode is not False and hasattr(_unicode, "__call__"):
80.141 + to_unicode = _unicode
80.142 +
80.143 + def unicodify(s):
80.144 + if _unicode and isinstance(s, str): return to_unicode(s)
80.145 + else: return s
80.146 +
80.147 + def getvalue(x):
80.148 + if hasattr(x, 'file') and hasattr(x, 'value'):
80.149 + return x.value
80.150 + elif hasattr(x, 'value'):
80.151 + return unicodify(x.value)
80.152 + else:
80.153 + return unicodify(x)
80.154 +
80.155 + stor = Storage()
80.156 + for key in requireds + tuple(mapping.keys()):
80.157 + value = mapping[key]
80.158 + if isinstance(value, list):
80.159 + if isinstance(defaults.get(key), list):
80.160 + value = [getvalue(x) for x in value]
80.161 + else:
80.162 + value = value[-1]
80.163 + if not isinstance(defaults.get(key), dict):
80.164 + value = getvalue(value)
80.165 + if isinstance(defaults.get(key), list) and not isinstance(value, list):
80.166 + value = [value]
80.167 + setattr(stor, key, value)
80.168 +
80.169 + for (key, value) in defaults.iteritems():
80.170 + result = value
80.171 + if hasattr(stor, key):
80.172 + result = stor[key]
80.173 + if value == () and not isinstance(result, tuple):
80.174 + result = (result,)
80.175 + setattr(stor, key, result)
80.176 +
80.177 + return stor
80.178 +
80.179 +class Counter(storage):
80.180 + """Keeps count of how many times something is added.
80.181 +
80.182 + >>> c = counter()
80.183 + >>> c.add('x')
80.184 + >>> c.add('x')
80.185 + >>> c.add('x')
80.186 + >>> c.add('x')
80.187 + >>> c.add('x')
80.188 + >>> c.add('y')
80.189 + >>> c
80.190 + <Counter {'y': 1, 'x': 5}>
80.191 + >>> c.most()
80.192 + ['x']
80.193 + """
80.194 + def add(self, n):
80.195 + self.setdefault(n, 0)
80.196 + self[n] += 1
80.197 +
80.198 + def most(self):
80.199 + """Returns the keys with maximum count."""
80.200 + m = max(self.itervalues())
80.201 + return [k for k, v in self.iteritems() if v == m]
80.202 +
80.203 + def least(self):
80.204 + """Returns the keys with mininum count."""
80.205 + m = min(self.itervalues())
80.206 + return [k for k, v in self.iteritems() if v == m]
80.207 +
80.208 + def percent(self, key):
80.209 + """Returns what percentage a certain key is of all entries.
80.210 +
80.211 + >>> c = counter()
80.212 + >>> c.add('x')
80.213 + >>> c.add('x')
80.214 + >>> c.add('x')
80.215 + >>> c.add('y')
80.216 + >>> c.percent('x')
80.217 + 0.75
80.218 + >>> c.percent('y')
80.219 + 0.25
80.220 + """
80.221 + return float(self[key])/sum(self.values())
80.222 +
80.223 + def sorted_keys(self):
80.224 + """Returns keys sorted by value.
80.225 +
80.226 + >>> c = counter()
80.227 + >>> c.add('x')
80.228 + >>> c.add('x')
80.229 + >>> c.add('y')
80.230 + >>> c.sorted_keys()
80.231 + ['x', 'y']
80.232 + """
80.233 + return sorted(self.keys(), key=lambda k: self[k], reverse=True)
80.234 +
80.235 + def sorted_values(self):
80.236 + """Returns values sorted by value.
80.237 +
80.238 + >>> c = counter()
80.239 + >>> c.add('x')
80.240 + >>> c.add('x')
80.241 + >>> c.add('y')
80.242 + >>> c.sorted_values()
80.243 + [2, 1]
80.244 + """
80.245 + return [self[k] for k in self.sorted_keys()]
80.246 +
80.247 + def sorted_items(self):
80.248 + """Returns items sorted by value.
80.249 +
80.250 + >>> c = counter()
80.251 + >>> c.add('x')
80.252 + >>> c.add('x')
80.253 + >>> c.add('y')
80.254 + >>> c.sorted_items()
80.255 + [('x', 2), ('y', 1)]
80.256 + """
80.257 + return [(k, self[k]) for k in self.sorted_keys()]
80.258 +
80.259 + def __repr__(self):
80.260 + return '<Counter ' + dict.__repr__(self) + '>'
80.261 +
80.262 +counter = Counter
80.263 +
80.264 +iters = [list, tuple]
80.265 +import __builtin__
80.266 +if hasattr(__builtin__, 'set'):
80.267 + iters.append(set)
80.268 +if hasattr(__builtin__, 'frozenset'):
80.269 + iters.append(set)
80.270 +if sys.version_info < (2,6): # sets module deprecated in 2.6
80.271 + try:
80.272 + from sets import Set
80.273 + iters.append(Set)
80.274 + except ImportError:
80.275 + pass
80.276 +
80.277 +class _hack(tuple): pass
80.278 +iters = _hack(iters)
80.279 +iters.__doc__ = """
80.280 +A list of iterable items (like lists, but not strings). Includes whichever
80.281 +of lists, tuples, sets, and Sets are available in this version of Python.
80.282 +"""
80.283 +
80.284 +def _strips(direction, text, remove):
80.285 + if isinstance(remove, iters):
80.286 + for subr in remove:
80.287 + text = _strips(direction, text, subr)
80.288 + return text
80.289 +
80.290 + if direction == 'l':
80.291 + if text.startswith(remove):
80.292 + return text[len(remove):]
80.293 + elif direction == 'r':
80.294 + if text.endswith(remove):
80.295 + return text[:-len(remove)]
80.296 + else:
80.297 + raise ValueError, "Direction needs to be r or l."
80.298 + return text
80.299 +
80.300 +def rstrips(text, remove):
80.301 + """
80.302 + removes the string `remove` from the right of `text`
80.303 +
80.304 + >>> rstrips("foobar", "bar")
80.305 + 'foo'
80.306 +
80.307 + """
80.308 + return _strips('r', text, remove)
80.309 +
80.310 +def lstrips(text, remove):
80.311 + """
80.312 + removes the string `remove` from the left of `text`
80.313 +
80.314 + >>> lstrips("foobar", "foo")
80.315 + 'bar'
80.316 + >>> lstrips('http://foo.org/', ['http://', 'https://'])
80.317 + 'foo.org/'
80.318 + >>> lstrips('FOOBARBAZ', ['FOO', 'BAR'])
80.319 + 'BAZ'
80.320 + >>> lstrips('FOOBARBAZ', ['BAR', 'FOO'])
80.321 + 'BARBAZ'
80.322 +
80.323 + """
80.324 + return _strips('l', text, remove)
80.325 +
80.326 +def strips(text, remove):
80.327 + """
80.328 + removes the string `remove` from the both sides of `text`
80.329 +
80.330 + >>> strips("foobarfoo", "foo")
80.331 + 'bar'
80.332 +
80.333 + """
80.334 + return rstrips(lstrips(text, remove), remove)
80.335 +
80.336 +def safeunicode(obj, encoding='utf-8'):
80.337 + r"""
80.338 + Converts any given object to unicode string.
80.339 +
80.340 + >>> safeunicode('hello')
80.341 + u'hello'
80.342 + >>> safeunicode(2)
80.343 + u'2'
80.344 + >>> safeunicode('\xe1\x88\xb4')
80.345 + u'\u1234'
80.346 + """
80.347 + t = type(obj)
80.348 + if t is unicode:
80.349 + return obj
80.350 + elif t is str:
80.351 + return obj.decode(encoding)
80.352 + elif t in [int, float, bool]:
80.353 + return unicode(obj)
80.354 + elif hasattr(obj, '__unicode__') or isinstance(obj, unicode):
80.355 + return unicode(obj)
80.356 + else:
80.357 + return str(obj).decode(encoding)
80.358 +
80.359 +def safestr(obj, encoding='utf-8'):
80.360 + r"""
80.361 + Converts any given object to utf-8 encoded string.
80.362 +
80.363 + >>> safestr('hello')
80.364 + 'hello'
80.365 + >>> safestr(u'\u1234')
80.366 + '\xe1\x88\xb4'
80.367 + >>> safestr(2)
80.368 + '2'
80.369 + """
80.370 + if isinstance(obj, unicode):
80.371 + return obj.encode(encoding)
80.372 + elif isinstance(obj, str):
80.373 + return obj
80.374 + elif hasattr(obj, 'next'): # iterator
80.375 + return itertools.imap(safestr, obj)
80.376 + else:
80.377 + return str(obj)
80.378 +
80.379 +# for backward-compatibility
80.380 +utf8 = safestr
80.381 +
80.382 +class TimeoutError(Exception): pass
80.383 +def timelimit(timeout):
80.384 + """
80.385 + A decorator to limit a function to `timeout` seconds, raising `TimeoutError`
80.386 + if it takes longer.
80.387 +
80.388 + >>> import time
80.389 + >>> def meaningoflife():
80.390 + ... time.sleep(.2)
80.391 + ... return 42
80.392 + >>>
80.393 + >>> timelimit(.1)(meaningoflife)()
80.394 + Traceback (most recent call last):
80.395 + ...
80.396 + TimeoutError: took too long
80.397 + >>> timelimit(1)(meaningoflife)()
80.398 + 42
80.399 +
80.400 + _Caveat:_ The function isn't stopped after `timeout` seconds but continues
80.401 + executing in a separate thread. (There seems to be no way to kill a thread.)
80.402 +
80.403 + inspired by <http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/473878>
80.404 + """
80.405 + def _1(function):
80.406 + def _2(*args, **kw):
80.407 + class Dispatch(threading.Thread):
80.408 + def __init__(self):
80.409 + threading.Thread.__init__(self)
80.410 + self.result = None
80.411 + self.error = None
80.412 +
80.413 + self.setDaemon(True)
80.414 + self.start()
80.415 +
80.416 + def run(self):
80.417 + try:
80.418 + self.result = function(*args, **kw)
80.419 + except:
80.420 + self.error = sys.exc_info()
80.421 +
80.422 + c = Dispatch()
80.423 + c.join(timeout)
80.424 + if c.isAlive():
80.425 + raise TimeoutError, 'took too long'
80.426 + if c.error:
80.427 + raise c.error[0], c.error[1]
80.428 + return c.result
80.429 + return _2
80.430 + return _1
80.431 +
80.432 +class Memoize:
80.433 + """
80.434 + 'Memoizes' a function, caching its return values for each input.
80.435 + If `expires` is specified, values are recalculated after `expires` seconds.
80.436 + If `background` is specified, values are recalculated in a separate thread.
80.437 +
80.438 + >>> calls = 0
80.439 + >>> def howmanytimeshaveibeencalled():
80.440 + ... global calls
80.441 + ... calls += 1
80.442 + ... return calls
80.443 + >>> fastcalls = memoize(howmanytimeshaveibeencalled)
80.444 + >>> howmanytimeshaveibeencalled()
80.445 + 1
80.446 + >>> howmanytimeshaveibeencalled()
80.447 + 2
80.448 + >>> fastcalls()
80.449 + 3
80.450 + >>> fastcalls()
80.451 + 3
80.452 + >>> import time
80.453 + >>> fastcalls = memoize(howmanytimeshaveibeencalled, .1, background=False)
80.454 + >>> fastcalls()
80.455 + 4
80.456 + >>> fastcalls()
80.457 + 4
80.458 + >>> time.sleep(.2)
80.459 + >>> fastcalls()
80.460 + 5
80.461 + >>> def slowfunc():
80.462 + ... time.sleep(.1)
80.463 + ... return howmanytimeshaveibeencalled()
80.464 + >>> fastcalls = memoize(slowfunc, .2, background=True)
80.465 + >>> fastcalls()
80.466 + 6
80.467 + >>> timelimit(.05)(fastcalls)()
80.468 + 6
80.469 + >>> time.sleep(.2)
80.470 + >>> timelimit(.05)(fastcalls)()
80.471 + 6
80.472 + >>> timelimit(.05)(fastcalls)()
80.473 + 6
80.474 + >>> time.sleep(.2)
80.475 + >>> timelimit(.05)(fastcalls)()
80.476 + 7
80.477 + >>> fastcalls = memoize(slowfunc, None, background=True)
80.478 + >>> threading.Thread(target=fastcalls).start()
80.479 + >>> time.sleep(.01)
80.480 + >>> fastcalls()
80.481 + 9
80.482 + """
80.483 + def __init__(self, func, expires=None, background=True):
80.484 + self.func = func
80.485 + self.cache = {}
80.486 + self.expires = expires
80.487 + self.background = background
80.488 + self.running = {}
80.489 +
80.490 + def __call__(self, *args, **keywords):
80.491 + key = (args, tuple(keywords.items()))
80.492 + if not self.running.get(key):
80.493 + self.running[key] = threading.Lock()
80.494 + def update(block=False):
80.495 + if self.running[key].acquire(block):
80.496 + try:
80.497 + self.cache[key] = (self.func(*args, **keywords), time.time())
80.498 + finally:
80.499 + self.running[key].release()
80.500 +
80.501 + if key not in self.cache:
80.502 + update(block=True)
80.503 + elif self.expires and (time.time() - self.cache[key][1]) > self.expires:
80.504 + if self.background:
80.505 + threading.Thread(target=update).start()
80.506 + else:
80.507 + update()
80.508 + return self.cache[key][0]
80.509 +
80.510 +memoize = Memoize
80.511 +
80.512 +re_compile = memoize(re.compile) #@@ threadsafe?
80.513 +re_compile.__doc__ = """
80.514 +A memoized version of re.compile.
80.515 +"""
80.516 +
80.517 +class _re_subm_proxy:
80.518 + def __init__(self):
80.519 + self.match = None
80.520 + def __call__(self, match):
80.521 + self.match = match
80.522 + return ''
80.523 +
80.524 +def re_subm(pat, repl, string):
80.525 + """
80.526 + Like re.sub, but returns the replacement _and_ the match object.
80.527 +
80.528 + >>> t, m = re_subm('g(oo+)fball', r'f\\1lish', 'goooooofball')
80.529 + >>> t
80.530 + 'foooooolish'
80.531 + >>> m.groups()
80.532 + ('oooooo',)
80.533 + """
80.534 + compiled_pat = re_compile(pat)
80.535 + proxy = _re_subm_proxy()
80.536 + compiled_pat.sub(proxy.__call__, string)
80.537 + return compiled_pat.sub(repl, string), proxy.match
80.538 +
80.539 +def group(seq, size):
80.540 + """
80.541 + Returns an iterator over a series of lists of length size from iterable.
80.542 +
80.543 + >>> list(group([1,2,3,4], 2))
80.544 + [[1, 2], [3, 4]]
80.545 + >>> list(group([1,2,3,4,5], 2))
80.546 + [[1, 2], [3, 4], [5]]
80.547 + """
80.548 + def take(seq, n):
80.549 + for i in xrange(n):
80.550 + yield seq.next()
80.551 +
80.552 + if not hasattr(seq, 'next'):
80.553 + seq = iter(seq)
80.554 + while True:
80.555 + x = list(take(seq, size))
80.556 + if x:
80.557 + yield x
80.558 + else:
80.559 + break
80.560 +
80.561 +def uniq(seq, key=None):
80.562 + """
80.563 + Removes duplicate elements from a list while preserving the order of the rest.
80.564 +
80.565 + >>> uniq([9,0,2,1,0])
80.566 + [9, 0, 2, 1]
80.567 +
80.568 + The value of the optional `key` parameter should be a function that
80.569 + takes a single argument and returns a key to test the uniqueness.
80.570 +
80.571 + >>> uniq(["Foo", "foo", "bar"], key=lambda s: s.lower())
80.572 + ['Foo', 'bar']
80.573 + """
80.574 + key = key or (lambda x: x)
80.575 + seen = set()
80.576 + result = []
80.577 + for v in seq:
80.578 + k = key(v)
80.579 + if k in seen:
80.580 + continue
80.581 + seen.add(k)
80.582 + result.append(v)
80.583 + return result
80.584 +
80.585 +def iterview(x):
80.586 + """
80.587 + Takes an iterable `x` and returns an iterator over it
80.588 + which prints its progress to stderr as it iterates through.
80.589 + """
80.590 + WIDTH = 70
80.591 +
80.592 + def plainformat(n, lenx):
80.593 + return '%5.1f%% (%*d/%d)' % ((float(n)/lenx)*100, len(str(lenx)), n, lenx)
80.594 +
80.595 + def bars(size, n, lenx):
80.596 + val = int((float(n)*size)/lenx + 0.5)
80.597 + if size - val:
80.598 + spacing = ">" + (" "*(size-val))[1:]
80.599 + else:
80.600 + spacing = ""
80.601 + return "[%s%s]" % ("="*val, spacing)
80.602 +
80.603 + def eta(elapsed, n, lenx):
80.604 + if n == 0:
80.605 + return '--:--:--'
80.606 + if n == lenx:
80.607 + secs = int(elapsed)
80.608 + else:
80.609 + secs = int((elapsed/n) * (lenx-n))
80.610 + mins, secs = divmod(secs, 60)
80.611 + hrs, mins = divmod(mins, 60)
80.612 +
80.613 + return '%02d:%02d:%02d' % (hrs, mins, secs)
80.614 +
80.615 + def format(starttime, n, lenx):
80.616 + out = plainformat(n, lenx) + ' '
80.617 + if n == lenx:
80.618 + end = ' '
80.619 + else:
80.620 + end = ' ETA '
80.621 + end += eta(time.time() - starttime, n, lenx)
80.622 + out += bars(WIDTH - len(out) - len(end), n, lenx)
80.623 + out += end
80.624 + return out
80.625 +
80.626 + starttime = time.time()
80.627 + lenx = len(x)
80.628 + for n, y in enumerate(x):
80.629 + sys.stderr.write('\r' + format(starttime, n, lenx))
80.630 + yield y
80.631 + sys.stderr.write('\r' + format(starttime, n+1, lenx) + '\n')
80.632 +
80.633 +class IterBetter:
80.634 + """
80.635 + Returns an object that can be used as an iterator
80.636 + but can also be used via __getitem__ (although it
80.637 + cannot go backwards -- that is, you cannot request
80.638 + `iterbetter[0]` after requesting `iterbetter[1]`).
80.639 +
80.640 + >>> import itertools
80.641 + >>> c = iterbetter(itertools.count())
80.642 + >>> c[1]
80.643 + 1
80.644 + >>> c[5]
80.645 + 5
80.646 + >>> c[3]
80.647 + Traceback (most recent call last):
80.648 + ...
80.649 + IndexError: already passed 3
80.650 +
80.651 + For boolean test, IterBetter peeps at first value in the itertor without effecting the iteration.
80.652 +
80.653 + >>> c = iterbetter(iter(range(5)))
80.654 + >>> bool(c)
80.655 + True
80.656 + >>> list(c)
80.657 + [0, 1, 2, 3, 4]
80.658 + >>> c = iterbetter(iter([]))
80.659 + >>> bool(c)
80.660 + False
80.661 + >>> list(c)
80.662 + []
80.663 + """
80.664 + def __init__(self, iterator):
80.665 + self.i, self.c = iterator, 0
80.666 +
80.667 + def __iter__(self):
80.668 + if hasattr(self, "_head"):
80.669 + yield self._head
80.670 +
80.671 + while 1:
80.672 + yield self.i.next()
80.673 + self.c += 1
80.674 +
80.675 + def __getitem__(self, i):
80.676 + #todo: slices
80.677 + if i < self.c:
80.678 + raise IndexError, "already passed "+str(i)
80.679 + try:
80.680 + while i > self.c:
80.681 + self.i.next()
80.682 + self.c += 1
80.683 + # now self.c == i
80.684 + self.c += 1
80.685 + return self.i.next()
80.686 + except StopIteration:
80.687 + raise IndexError, str(i)
80.688 +
80.689 + def __nonzero__(self):
80.690 + if hasattr(self, "__len__"):
80.691 + return len(self) != 0
80.692 + elif hasattr(self, "_head"):
80.693 + return True
80.694 + else:
80.695 + try:
80.696 + self._head = self.i.next()
80.697 + except StopIteration:
80.698 + return False
80.699 + else:
80.700 + return True
80.701 +
80.702 +iterbetter = IterBetter
80.703 +
80.704 +def safeiter(it, cleanup=None, ignore_errors=True):
80.705 + """Makes an iterator safe by ignoring the exceptions occured during the iteration.
80.706 + """
80.707 + def next():
80.708 + while True:
80.709 + try:
80.710 + return it.next()
80.711 + except StopIteration:
80.712 + raise
80.713 + except:
80.714 + traceback.print_exc()
80.715 +
80.716 + it = iter(it)
80.717 + while True:
80.718 + yield next()
80.719 +
80.720 +def safewrite(filename, content):
80.721 + """Writes the content to a temp file and then moves the temp file to
80.722 + given filename to avoid overwriting the existing file in case of errors.
80.723 + """
80.724 + f = file(filename + '.tmp', 'w')
80.725 + f.write(content)
80.726 + f.close()
80.727 + os.rename(f.name, filename)
80.728 +
80.729 +def dictreverse(mapping):
80.730 + """
80.731 + Returns a new dictionary with keys and values swapped.
80.732 +
80.733 + >>> dictreverse({1: 2, 3: 4})
80.734 + {2: 1, 4: 3}
80.735 + """
80.736 + return dict([(value, key) for (key, value) in mapping.iteritems()])
80.737 +
80.738 +def dictfind(dictionary, element):
80.739 + """
80.740 + Returns a key whose value in `dictionary` is `element`
80.741 + or, if none exists, None.
80.742 +
80.743 + >>> d = {1:2, 3:4}
80.744 + >>> dictfind(d, 4)
80.745 + 3
80.746 + >>> dictfind(d, 5)
80.747 + """
80.748 + for (key, value) in dictionary.iteritems():
80.749 + if element is value:
80.750 + return key
80.751 +
80.752 +def dictfindall(dictionary, element):
80.753 + """
80.754 + Returns the keys whose values in `dictionary` are `element`
80.755 + or, if none exists, [].
80.756 +
80.757 + >>> d = {1:4, 3:4}
80.758 + >>> dictfindall(d, 4)
80.759 + [1, 3]
80.760 + >>> dictfindall(d, 5)
80.761 + []
80.762 + """
80.763 + res = []
80.764 + for (key, value) in dictionary.iteritems():
80.765 + if element is value:
80.766 + res.append(key)
80.767 + return res
80.768 +
80.769 +def dictincr(dictionary, element):
80.770 + """
80.771 + Increments `element` in `dictionary`,
80.772 + setting it to one if it doesn't exist.
80.773 +
80.774 + >>> d = {1:2, 3:4}
80.775 + >>> dictincr(d, 1)
80.776 + 3
80.777 + >>> d[1]
80.778 + 3
80.779 + >>> dictincr(d, 5)
80.780 + 1
80.781 + >>> d[5]
80.782 + 1
80.783 + """
80.784 + dictionary.setdefault(element, 0)
80.785 + dictionary[element] += 1
80.786 + return dictionary[element]
80.787 +
80.788 +def dictadd(*dicts):
80.789 + """
80.790 + Returns a dictionary consisting of the keys in the argument dictionaries.
80.791 + If they share a key, the value from the last argument is used.
80.792 +
80.793 + >>> dictadd({1: 0, 2: 0}, {2: 1, 3: 1})
80.794 + {1: 0, 2: 1, 3: 1}
80.795 + """
80.796 + result = {}
80.797 + for dct in dicts:
80.798 + result.update(dct)
80.799 + return result
80.800 +
80.801 +def requeue(queue, index=-1):
80.802 + """Returns the element at index after moving it to the beginning of the queue.
80.803 +
80.804 + >>> x = [1, 2, 3, 4]
80.805 + >>> requeue(x)
80.806 + 4
80.807 + >>> x
80.808 + [4, 1, 2, 3]
80.809 + """
80.810 + x = queue.pop(index)
80.811 + queue.insert(0, x)
80.812 + return x
80.813 +
80.814 +def restack(stack, index=0):
80.815 + """Returns the element at index after moving it to the top of stack.
80.816 +
80.817 + >>> x = [1, 2, 3, 4]
80.818 + >>> restack(x)
80.819 + 1
80.820 + >>> x
80.821 + [2, 3, 4, 1]
80.822 + """
80.823 + x = stack.pop(index)
80.824 + stack.append(x)
80.825 + return x
80.826 +
80.827 +def listget(lst, ind, default=None):
80.828 + """
80.829 + Returns `lst[ind]` if it exists, `default` otherwise.
80.830 +
80.831 + >>> listget(['a'], 0)
80.832 + 'a'
80.833 + >>> listget(['a'], 1)
80.834 + >>> listget(['a'], 1, 'b')
80.835 + 'b'
80.836 + """
80.837 + if len(lst)-1 < ind:
80.838 + return default
80.839 + return lst[ind]
80.840 +
80.841 +def intget(integer, default=None):
80.842 + """
80.843 + Returns `integer` as an int or `default` if it can't.
80.844 +
80.845 + >>> intget('3')
80.846 + 3
80.847 + >>> intget('3a')
80.848 + >>> intget('3a', 0)
80.849 + 0
80.850 + """
80.851 + try:
80.852 + return int(integer)
80.853 + except (TypeError, ValueError):
80.854 + return default
80.855 +
80.856 +def datestr(then, now=None):
80.857 + """
80.858 + Converts a (UTC) datetime object to a nice string representation.
80.859 +
80.860 + >>> from datetime import datetime, timedelta
80.861 + >>> d = datetime(1970, 5, 1)
80.862 + >>> datestr(d, now=d)
80.863 + '0 microseconds ago'
80.864 + >>> for t, v in {
80.865 + ... timedelta(microseconds=1): '1 microsecond ago',
80.866 + ... timedelta(microseconds=2): '2 microseconds ago',
80.867 + ... -timedelta(microseconds=1): '1 microsecond from now',
80.868 + ... -timedelta(microseconds=2): '2 microseconds from now',
80.869 + ... timedelta(microseconds=2000): '2 milliseconds ago',
80.870 + ... timedelta(seconds=2): '2 seconds ago',
80.871 + ... timedelta(seconds=2*60): '2 minutes ago',
80.872 + ... timedelta(seconds=2*60*60): '2 hours ago',
80.873 + ... timedelta(days=2): '2 days ago',
80.874 + ... }.iteritems():
80.875 + ... assert datestr(d, now=d+t) == v
80.876 + >>> datestr(datetime(1970, 1, 1), now=d)
80.877 + 'January 1'
80.878 + >>> datestr(datetime(1969, 1, 1), now=d)
80.879 + 'January 1, 1969'
80.880 + >>> datestr(datetime(1970, 6, 1), now=d)
80.881 + 'June 1, 1970'
80.882 + >>> datestr(None)
80.883 + ''
80.884 + """
80.885 + def agohence(n, what, divisor=None):
80.886 + if divisor: n = n // divisor
80.887 +
80.888 + out = str(abs(n)) + ' ' + what # '2 day'
80.889 + if abs(n) != 1: out += 's' # '2 days'
80.890 + out += ' ' # '2 days '
80.891 + if n < 0:
80.892 + out += 'from now'
80.893 + else:
80.894 + out += 'ago'
80.895 + return out # '2 days ago'
80.896 +
80.897 + oneday = 24 * 60 * 60
80.898 +
80.899 + if not then: return ""
80.900 + if not now: now = datetime.datetime.utcnow()
80.901 + if type(now).__name__ == "DateTime":
80.902 + now = datetime.datetime.fromtimestamp(now)
80.903 + if type(then).__name__ == "DateTime":
80.904 + then = datetime.datetime.fromtimestamp(then)
80.905 + elif type(then).__name__ == "date":
80.906 + then = datetime.datetime(then.year, then.month, then.day)
80.907 +
80.908 + delta = now - then
80.909 + deltaseconds = int(delta.days * oneday + delta.seconds + delta.microseconds * 1e-06)
80.910 + deltadays = abs(deltaseconds) // oneday
80.911 + if deltaseconds < 0: deltadays *= -1 # fix for oddity of floor
80.912 +
80.913 + if deltadays:
80.914 + if abs(deltadays) < 4:
80.915 + return agohence(deltadays, 'day')
80.916 +
80.917 + try:
80.918 + out = then.strftime('%B %e') # e.g. 'June 3'
80.919 + except ValueError:
80.920 + # %e doesn't work on Windows.
80.921 + out = then.strftime('%B %d') # e.g. 'June 03'
80.922 +
80.923 + if then.year != now.year or deltadays < 0:
80.924 + out += ', %s' % then.year
80.925 + return out
80.926 +
80.927 + if int(deltaseconds):
80.928 + if abs(deltaseconds) > (60 * 60):
80.929 + return agohence(deltaseconds, 'hour', 60 * 60)
80.930 + elif abs(deltaseconds) > 60:
80.931 + return agohence(deltaseconds, 'minute', 60)
80.932 + else:
80.933 + return agohence(deltaseconds, 'second')
80.934 +
80.935 + deltamicroseconds = delta.microseconds
80.936 + if delta.days: deltamicroseconds = int(delta.microseconds - 1e6) # datetime oddity
80.937 + if abs(deltamicroseconds) > 1000:
80.938 + return agohence(deltamicroseconds, 'millisecond', 1000)
80.939 +
80.940 + return agohence(deltamicroseconds, 'microsecond')
80.941 +
80.942 +def numify(string):
80.943 + """
80.944 + Removes all non-digit characters from `string`.
80.945 +
80.946 + >>> numify('800-555-1212')
80.947 + '8005551212'
80.948 + >>> numify('800.555.1212')
80.949 + '8005551212'
80.950 +
80.951 + """
80.952 + return ''.join([c for c in str(string) if c.isdigit()])
80.953 +
80.954 +def denumify(string, pattern):
80.955 + """
80.956 + Formats `string` according to `pattern`, where the letter X gets replaced
80.957 + by characters from `string`.
80.958 +
80.959 + >>> denumify("8005551212", "(XXX) XXX-XXXX")
80.960 + '(800) 555-1212'
80.961 +
80.962 + """
80.963 + out = []
80.964 + for c in pattern:
80.965 + if c == "X":
80.966 + out.append(string[0])
80.967 + string = string[1:]
80.968 + else:
80.969 + out.append(c)
80.970 + return ''.join(out)
80.971 +
80.972 +def commify(n):
80.973 + """
80.974 + Add commas to an integer `n`.
80.975 +
80.976 + >>> commify(1)
80.977 + '1'
80.978 + >>> commify(123)
80.979 + '123'
80.980 + >>> commify(1234)
80.981 + '1,234'
80.982 + >>> commify(1234567890)
80.983 + '1,234,567,890'
80.984 + >>> commify(123.0)
80.985 + '123.0'
80.986 + >>> commify(1234.5)
80.987 + '1,234.5'
80.988 + >>> commify(1234.56789)
80.989 + '1,234.56789'
80.990 + >>> commify('%.2f' % 1234.5)
80.991 + '1,234.50'
80.992 + >>> commify(None)
80.993 + >>>
80.994 +
80.995 + """
80.996 + if n is None: return None
80.997 + n = str(n)
80.998 + if '.' in n:
80.999 + dollars, cents = n.split('.')
80.1000 + else:
80.1001 + dollars, cents = n, None
80.1002 +
80.1003 + r = []
80.1004 + for i, c in enumerate(str(dollars)[::-1]):
80.1005 + if i and (not (i % 3)):
80.1006 + r.insert(0, ',')
80.1007 + r.insert(0, c)
80.1008 + out = ''.join(r)
80.1009 + if cents:
80.1010 + out += '.' + cents
80.1011 + return out
80.1012 +
80.1013 +def dateify(datestring):
80.1014 + """
80.1015 + Formats a numified `datestring` properly.
80.1016 + """
80.1017 + return denumify(datestring, "XXXX-XX-XX XX:XX:XX")
80.1018 +
80.1019 +
80.1020 +def nthstr(n):
80.1021 + """
80.1022 + Formats an ordinal.
80.1023 + Doesn't handle negative numbers.
80.1024 +
80.1025 + >>> nthstr(1)
80.1026 + '1st'
80.1027 + >>> nthstr(0)
80.1028 + '0th'
80.1029 + >>> [nthstr(x) for x in [2, 3, 4, 5, 10, 11, 12, 13, 14, 15]]
80.1030 + ['2nd', '3rd', '4th', '5th', '10th', '11th', '12th', '13th', '14th', '15th']
80.1031 + >>> [nthstr(x) for x in [91, 92, 93, 94, 99, 100, 101, 102]]
80.1032 + ['91st', '92nd', '93rd', '94th', '99th', '100th', '101st', '102nd']
80.1033 + >>> [nthstr(x) for x in [111, 112, 113, 114, 115]]
80.1034 + ['111th', '112th', '113th', '114th', '115th']
80.1035 +
80.1036 + """
80.1037 +
80.1038 + assert n >= 0
80.1039 + if n % 100 in [11, 12, 13]: return '%sth' % n
80.1040 + return {1: '%sst', 2: '%snd', 3: '%srd'}.get(n % 10, '%sth') % n
80.1041 +
80.1042 +def cond(predicate, consequence, alternative=None):
80.1043 + """
80.1044 + Function replacement for if-else to use in expressions.
80.1045 +
80.1046 + >>> x = 2
80.1047 + >>> cond(x % 2 == 0, "even", "odd")
80.1048 + 'even'
80.1049 + >>> cond(x % 2 == 0, "even", "odd") + '_row'
80.1050 + 'even_row'
80.1051 + """
80.1052 + if predicate:
80.1053 + return consequence
80.1054 + else:
80.1055 + return alternative
80.1056 +
80.1057 +class CaptureStdout:
80.1058 + """
80.1059 + Captures everything `func` prints to stdout and returns it instead.
80.1060 +
80.1061 + >>> def idiot():
80.1062 + ... print "foo"
80.1063 + >>> capturestdout(idiot)()
80.1064 + 'foo\\n'
80.1065 +
80.1066 + **WARNING:** Not threadsafe!
80.1067 + """
80.1068 + def __init__(self, func):
80.1069 + self.func = func
80.1070 + def __call__(self, *args, **keywords):
80.1071 + from cStringIO import StringIO
80.1072 + # Not threadsafe!
80.1073 + out = StringIO()
80.1074 + oldstdout = sys.stdout
80.1075 + sys.stdout = out
80.1076 + try:
80.1077 + self.func(*args, **keywords)
80.1078 + finally:
80.1079 + sys.stdout = oldstdout
80.1080 + return out.getvalue()
80.1081 +
80.1082 +capturestdout = CaptureStdout
80.1083 +
80.1084 +class Profile:
80.1085 + """
80.1086 + Profiles `func` and returns a tuple containing its output
80.1087 + and a string with human-readable profiling information.
80.1088 +
80.1089 + >>> import time
80.1090 + >>> out, inf = profile(time.sleep)(.001)
80.1091 + >>> out
80.1092 + >>> inf[:10].strip()
80.1093 + 'took 0.0'
80.1094 + """
80.1095 + def __init__(self, func):
80.1096 + self.func = func
80.1097 + def __call__(self, *args): ##, **kw): kw unused
80.1098 + import hotshot, hotshot.stats, os, tempfile ##, time already imported
80.1099 + f, filename = tempfile.mkstemp()
80.1100 + os.close(f)
80.1101 +
80.1102 + prof = hotshot.Profile(filename)
80.1103 +
80.1104 + stime = time.time()
80.1105 + result = prof.runcall(self.func, *args)
80.1106 + stime = time.time() - stime
80.1107 + prof.close()
80.1108 +
80.1109 + import cStringIO
80.1110 + out = cStringIO.StringIO()
80.1111 + stats = hotshot.stats.load(filename)
80.1112 + stats.stream = out
80.1113 + stats.strip_dirs()
80.1114 + stats.sort_stats('time', 'calls')
80.1115 + stats.print_stats(40)
80.1116 + stats.print_callers()
80.1117 +
80.1118 + x = '\n\ntook '+ str(stime) + ' seconds\n'
80.1119 + x += out.getvalue()
80.1120 +
80.1121 + # remove the tempfile
80.1122 + try:
80.1123 + os.remove(filename)
80.1124 + except IOError:
80.1125 + pass
80.1126 +
80.1127 + return result, x
80.1128 +
80.1129 +profile = Profile
80.1130 +
80.1131 +
80.1132 +import traceback
80.1133 +# hack for compatibility with Python 2.3:
80.1134 +if not hasattr(traceback, 'format_exc'):
80.1135 + from cStringIO import StringIO
80.1136 + def format_exc(limit=None):
80.1137 + strbuf = StringIO()
80.1138 + traceback.print_exc(limit, strbuf)
80.1139 + return strbuf.getvalue()
80.1140 + traceback.format_exc = format_exc
80.1141 +
80.1142 +def tryall(context, prefix=None):
80.1143 + """
80.1144 + Tries a series of functions and prints their results.
80.1145 + `context` is a dictionary mapping names to values;
80.1146 + the value will only be tried if it's callable.
80.1147 +
80.1148 + >>> tryall(dict(j=lambda: True))
80.1149 + j: True
80.1150 + ----------------------------------------
80.1151 + results:
80.1152 + True: 1
80.1153 +
80.1154 + For example, you might have a file `test/stuff.py`
80.1155 + with a series of functions testing various things in it.
80.1156 + At the bottom, have a line:
80.1157 +
80.1158 + if __name__ == "__main__": tryall(globals())
80.1159 +
80.1160 + Then you can run `python test/stuff.py` and get the results of
80.1161 + all the tests.
80.1162 + """
80.1163 + context = context.copy() # vars() would update
80.1164 + results = {}
80.1165 + for (key, value) in context.iteritems():
80.1166 + if not hasattr(value, '__call__'):
80.1167 + continue
80.1168 + if prefix and not key.startswith(prefix):
80.1169 + continue
80.1170 + print key + ':',
80.1171 + try:
80.1172 + r = value()
80.1173 + dictincr(results, r)
80.1174 + print r
80.1175 + except:
80.1176 + print 'ERROR'
80.1177 + dictincr(results, 'ERROR')
80.1178 + print ' ' + '\n '.join(traceback.format_exc().split('\n'))
80.1179 +
80.1180 + print '-'*40
80.1181 + print 'results:'
80.1182 + for (key, value) in results.iteritems():
80.1183 + print ' '*2, str(key)+':', value
80.1184 +
80.1185 +class ThreadedDict(threadlocal):
80.1186 + """
80.1187 + Thread local storage.
80.1188 +
80.1189 + >>> d = ThreadedDict()
80.1190 + >>> d.x = 1
80.1191 + >>> d.x
80.1192 + 1
80.1193 + >>> import threading
80.1194 + >>> def f(): d.x = 2
80.1195 + ...
80.1196 + >>> t = threading.Thread(target=f)
80.1197 + >>> t.start()
80.1198 + >>> t.join()
80.1199 + >>> d.x
80.1200 + 1
80.1201 + """
80.1202 + _instances = set()
80.1203 +
80.1204 + def __init__(self):
80.1205 + ThreadedDict._instances.add(self)
80.1206 +
80.1207 + def __del__(self):
80.1208 + ThreadedDict._instances.remove(self)
80.1209 +
80.1210 + def __hash__(self):
80.1211 + return id(self)
80.1212 +
80.1213 + def clear_all():
80.1214 + """Clears all ThreadedDict instances.
80.1215 + """
80.1216 + for t in list(ThreadedDict._instances):
80.1217 + t.clear()
80.1218 + clear_all = staticmethod(clear_all)
80.1219 +
80.1220 + # Define all these methods to more or less fully emulate dict -- attribute access
80.1221 + # is built into threading.local.
80.1222 +
80.1223 + def __getitem__(self, key):
80.1224 + return self.__dict__[key]
80.1225 +
80.1226 + def __setitem__(self, key, value):
80.1227 + self.__dict__[key] = value
80.1228 +
80.1229 + def __delitem__(self, key):
80.1230 + del self.__dict__[key]
80.1231 +
80.1232 + def __contains__(self, key):
80.1233 + return key in self.__dict__
80.1234 +
80.1235 + has_key = __contains__
80.1236 +
80.1237 + def clear(self):
80.1238 + self.__dict__.clear()
80.1239 +
80.1240 + def copy(self):
80.1241 + return self.__dict__.copy()
80.1242 +
80.1243 + def get(self, key, default=None):
80.1244 + return self.__dict__.get(key, default)
80.1245 +
80.1246 + def items(self):
80.1247 + return self.__dict__.items()
80.1248 +
80.1249 + def iteritems(self):
80.1250 + return self.__dict__.iteritems()
80.1251 +
80.1252 + def keys(self):
80.1253 + return self.__dict__.keys()
80.1254 +
80.1255 + def iterkeys(self):
80.1256 + return self.__dict__.iterkeys()
80.1257 +
80.1258 + iter = iterkeys
80.1259 +
80.1260 + def values(self):
80.1261 + return self.__dict__.values()
80.1262 +
80.1263 + def itervalues(self):
80.1264 + return self.__dict__.itervalues()
80.1265 +
80.1266 + def pop(self, key, *args):
80.1267 + return self.__dict__.pop(key, *args)
80.1268 +
80.1269 + def popitem(self):
80.1270 + return self.__dict__.popitem()
80.1271 +
80.1272 + def setdefault(self, key, default=None):
80.1273 + return self.__dict__.setdefault(key, default)
80.1274 +
80.1275 + def update(self, *args, **kwargs):
80.1276 + self.__dict__.update(*args, **kwargs)
80.1277 +
80.1278 + def __repr__(self):
80.1279 + return '<ThreadedDict %r>' % self.__dict__
80.1280 +
80.1281 + __str__ = __repr__
80.1282 +
80.1283 +threadeddict = ThreadedDict
80.1284 +
80.1285 +def autoassign(self, locals):
80.1286 + """
80.1287 + Automatically assigns local variables to `self`.
80.1288 +
80.1289 + >>> self = storage()
80.1290 + >>> autoassign(self, dict(a=1, b=2))
80.1291 + >>> self
80.1292 + <Storage {'a': 1, 'b': 2}>
80.1293 +
80.1294 + Generally used in `__init__` methods, as in:
80.1295 +
80.1296 + def __init__(self, foo, bar, baz=1): autoassign(self, locals())
80.1297 + """
80.1298 + for (key, value) in locals.iteritems():
80.1299 + if key == 'self':
80.1300 + continue
80.1301 + setattr(self, key, value)
80.1302 +
80.1303 +def to36(q):
80.1304 + """
80.1305 + Converts an integer to base 36 (a useful scheme for human-sayable IDs).
80.1306 +
80.1307 + >>> to36(35)
80.1308 + 'z'
80.1309 + >>> to36(119292)
80.1310 + '2k1o'
80.1311 + >>> int(to36(939387374), 36)
80.1312 + 939387374
80.1313 + >>> to36(0)
80.1314 + '0'
80.1315 + >>> to36(-393)
80.1316 + Traceback (most recent call last):
80.1317 + ...
80.1318 + ValueError: must supply a positive integer
80.1319 +
80.1320 + """
80.1321 + if q < 0: raise ValueError, "must supply a positive integer"
80.1322 + letters = "0123456789abcdefghijklmnopqrstuvwxyz"
80.1323 + converted = []
80.1324 + while q != 0:
80.1325 + q, r = divmod(q, 36)
80.1326 + converted.insert(0, letters[r])
80.1327 + return "".join(converted) or '0'
80.1328 +
80.1329 +
80.1330 +r_url = re_compile('(?<!\()(http://(\S+))')
80.1331 +def safemarkdown(text):
80.1332 + """
80.1333 + Converts text to HTML following the rules of Markdown, but blocking any
80.1334 + outside HTML input, so that only the things supported by Markdown
80.1335 + can be used. Also converts raw URLs to links.
80.1336 +
80.1337 + (requires [markdown.py](http://webpy.org/markdown.py))
80.1338 + """
80.1339 + from markdown import markdown
80.1340 + if text:
80.1341 + text = text.replace('<', '<')
80.1342 + # TODO: automatically get page title?
80.1343 + text = r_url.sub(r'<\1>', text)
80.1344 + text = markdown(text)
80.1345 + return text
80.1346 +
80.1347 +def sendmail(from_address, to_address, subject, message, headers=None, **kw):
80.1348 + """
80.1349 + Sends the email message `message` with mail and envelope headers
80.1350 + for from `from_address_` to `to_address` with `subject`.
80.1351 + Additional email headers can be specified with the dictionary
80.1352 + `headers.
80.1353 +
80.1354 + Optionally cc, bcc and attachments can be specified as keyword arguments.
80.1355 + Attachments must be an iterable and each attachment can be either a
80.1356 + filename or a file object or a dictionary with filename, content and
80.1357 + optionally content_type keys.
80.1358 +
80.1359 + If `web.config.smtp_server` is set, it will send the message
80.1360 + to that SMTP server. Otherwise it will look for
80.1361 + `/usr/sbin/sendmail`, the typical location for the sendmail-style
80.1362 + binary. To use sendmail from a different path, set `web.config.sendmail_path`.
80.1363 + """
80.1364 + attachments = kw.pop("attachments", [])
80.1365 + mail = _EmailMessage(from_address, to_address, subject, message, headers, **kw)
80.1366 +
80.1367 + for a in attachments:
80.1368 + if isinstance(a, dict):
80.1369 + mail.attach(a['filename'], a['content'], a.get('content_type'))
80.1370 + elif hasattr(a, 'read'): # file
80.1371 + filename = os.path.basename(getattr(a, "name", ""))
80.1372 + content_type = getattr(a, 'content_type', None)
80.1373 + mail.attach(filename, a.read(), content_type)
80.1374 + elif isinstance(a, basestring):
80.1375 + f = open(a, 'rb')
80.1376 + content = f.read()
80.1377 + f.close()
80.1378 + filename = os.path.basename(a)
80.1379 + mail.attach(filename, content, None)
80.1380 + else:
80.1381 + raise ValueError, "Invalid attachment: %s" % repr(a)
80.1382 +
80.1383 + mail.send()
80.1384 +
80.1385 +class _EmailMessage:
80.1386 + def __init__(self, from_address, to_address, subject, message, headers=None, **kw):
80.1387 + def listify(x):
80.1388 + if not isinstance(x, list):
80.1389 + return [safestr(x)]
80.1390 + else:
80.1391 + return [safestr(a) for a in x]
80.1392 +
80.1393 + subject = safestr(subject)
80.1394 + message = safestr(message)
80.1395 +
80.1396 + from_address = safestr(from_address)
80.1397 + to_address = listify(to_address)
80.1398 + cc = listify(kw.get('cc', []))
80.1399 + bcc = listify(kw.get('bcc', []))
80.1400 + recipients = to_address + cc + bcc
80.1401 +
80.1402 + import email.Utils
80.1403 + self.from_address = email.Utils.parseaddr(from_address)[1]
80.1404 + self.recipients = [email.Utils.parseaddr(r)[1] for r in recipients]
80.1405 +
80.1406 + self.headers = dictadd({
80.1407 + 'From': from_address,
80.1408 + 'To': ", ".join(to_address),
80.1409 + 'Subject': subject
80.1410 + }, headers or {})
80.1411 +
80.1412 + if cc:
80.1413 + self.headers['Cc'] = ", ".join(cc)
80.1414 +
80.1415 + self.message = self.new_message()
80.1416 + self.message.add_header("Content-Transfer-Encoding", "7bit")
80.1417 + self.message.add_header("Content-Disposition", "inline")
80.1418 + self.message.add_header("MIME-Version", "1.0")
80.1419 + self.message.set_payload(message, 'utf-8')
80.1420 + self.multipart = False
80.1421 +
80.1422 + def new_message(self):
80.1423 + from email.Message import Message
80.1424 + return Message()
80.1425 +
80.1426 + def attach(self, filename, content, content_type=None):
80.1427 + if not self.multipart:
80.1428 + msg = self.new_message()
80.1429 + msg.add_header("Content-Type", "multipart/mixed")
80.1430 + msg.attach(self.message)
80.1431 + self.message = msg
80.1432 + self.multipart = True
80.1433 +
80.1434 + import mimetypes
80.1435 + try:
80.1436 + from email import encoders
80.1437 + except:
80.1438 + from email import Encoders as encoders
80.1439 +
80.1440 + content_type = content_type or mimetypes.guess_type(filename)[0] or "applcation/octet-stream"
80.1441 +
80.1442 + msg = self.new_message()
80.1443 + msg.set_payload(content)
80.1444 + msg.add_header('Content-Type', content_type)
80.1445 + msg.add_header('Content-Disposition', 'attachment', filename=filename)
80.1446 +
80.1447 + if not content_type.startswith("text/"):
80.1448 + encoders.encode_base64(msg)
80.1449 +
80.1450 + self.message.attach(msg)
80.1451 +
80.1452 + def prepare_message(self):
80.1453 + for k, v in self.headers.iteritems():
80.1454 + if k.lower() == "content-type":
80.1455 + self.message.set_type(v)
80.1456 + else:
80.1457 + self.message.add_header(k, v)
80.1458 +
80.1459 + self.headers = {}
80.1460 +
80.1461 + def send(self):
80.1462 + try:
80.1463 + import webapi
80.1464 + except ImportError:
80.1465 + webapi = Storage(config=Storage())
80.1466 +
80.1467 + self.prepare_message()
80.1468 + message_text = self.message.as_string()
80.1469 +
80.1470 + if webapi.config.get('smtp_server'):
80.1471 + server = webapi.config.get('smtp_server')
80.1472 + port = webapi.config.get('smtp_port', 0)
80.1473 + username = webapi.config.get('smtp_username')
80.1474 + password = webapi.config.get('smtp_password')
80.1475 + debug_level = webapi.config.get('smtp_debuglevel', None)
80.1476 + starttls = webapi.config.get('smtp_starttls', False)
80.1477 +
80.1478 + import smtplib
80.1479 + smtpserver = smtplib.SMTP(server, port)
80.1480 +
80.1481 + if debug_level:
80.1482 + smtpserver.set_debuglevel(debug_level)
80.1483 +
80.1484 + if starttls:
80.1485 + smtpserver.ehlo()
80.1486 + smtpserver.starttls()
80.1487 + smtpserver.ehlo()
80.1488 +
80.1489 + if username and password:
80.1490 + smtpserver.login(username, password)
80.1491 +
80.1492 + smtpserver.sendmail(self.from_address, self.recipients, message_text)
80.1493 + smtpserver.quit()
80.1494 + elif webapi.config.get('email_engine') == 'aws':
80.1495 + import boto.ses
80.1496 + c = boto.ses.SESConnection(
80.1497 + aws_access_key_id=webapi.config.get('aws_access_key_id'),
80.1498 + aws_secret_access_key=web.api.config.get('aws_secret_access_key'))
80.1499 + c.send_raw_email(self.from_address, message_text, self.from_recipients)
80.1500 + else:
80.1501 + sendmail = webapi.config.get('sendmail_path', '/usr/sbin/sendmail')
80.1502 +
80.1503 + assert not self.from_address.startswith('-'), 'security'
80.1504 + for r in self.recipients:
80.1505 + assert not r.startswith('-'), 'security'
80.1506 +
80.1507 + cmd = [sendmail, '-f', self.from_address] + self.recipients
80.1508 +
80.1509 + if subprocess:
80.1510 + p = subprocess.Popen(cmd, stdin=subprocess.PIPE)
80.1511 + p.stdin.write(message_text)
80.1512 + p.stdin.close()
80.1513 + p.wait()
80.1514 + else:
80.1515 + i, o = os.popen2(cmd)
80.1516 + i.write(message)
80.1517 + i.close()
80.1518 + o.close()
80.1519 + del i, o
80.1520 +
80.1521 + def __repr__(self):
80.1522 + return "<EmailMessage>"
80.1523 +
80.1524 + def __str__(self):
80.1525 + return self.message.as_string()
80.1526 +
80.1527 +if __name__ == "__main__":
80.1528 + import doctest
80.1529 + doctest.testmod()
81.1 Binary file OpenSecurity/install/web.py-0.37/web/utils.pyc has changed
82.1 --- /dev/null Thu Jan 01 00:00:00 1970 +0000
82.2 +++ b/OpenSecurity/install/web.py-0.37/web/webapi.py Mon Dec 02 14:02:05 2013 +0100
82.3 @@ -0,0 +1,525 @@
82.4 +"""
82.5 +Web API (wrapper around WSGI)
82.6 +(from web.py)
82.7 +"""
82.8 +
82.9 +__all__ = [
82.10 + "config",
82.11 + "header", "debug",
82.12 + "input", "data",
82.13 + "setcookie", "cookies",
82.14 + "ctx",
82.15 + "HTTPError",
82.16 +
82.17 + # 200, 201, 202
82.18 + "OK", "Created", "Accepted",
82.19 + "ok", "created", "accepted",
82.20 +
82.21 + # 301, 302, 303, 304, 307
82.22 + "Redirect", "Found", "SeeOther", "NotModified", "TempRedirect",
82.23 + "redirect", "found", "seeother", "notmodified", "tempredirect",
82.24 +
82.25 + # 400, 401, 403, 404, 405, 406, 409, 410, 412, 415
82.26 + "BadRequest", "Unauthorized", "Forbidden", "NotFound", "NoMethod", "NotAcceptable", "Conflict", "Gone", "PreconditionFailed", "UnsupportedMediaType",
82.27 + "badrequest", "unauthorized", "forbidden", "notfound", "nomethod", "notacceptable", "conflict", "gone", "preconditionfailed", "unsupportedmediatype",
82.28 +
82.29 + # 500
82.30 + "InternalError",
82.31 + "internalerror",
82.32 +]
82.33 +
82.34 +import sys, cgi, Cookie, pprint, urlparse, urllib
82.35 +from utils import storage, storify, threadeddict, dictadd, intget, safestr
82.36 +
82.37 +config = storage()
82.38 +config.__doc__ = """
82.39 +A configuration object for various aspects of web.py.
82.40 +
82.41 +`debug`
82.42 + : when True, enables reloading, disabled template caching and sets internalerror to debugerror.
82.43 +"""
82.44 +
82.45 +class HTTPError(Exception):
82.46 + def __init__(self, status, headers={}, data=""):
82.47 + ctx.status = status
82.48 + for k, v in headers.items():
82.49 + header(k, v)
82.50 + self.data = data
82.51 + Exception.__init__(self, status)
82.52 +
82.53 +def _status_code(status, data=None, classname=None, docstring=None):
82.54 + if data is None:
82.55 + data = status.split(" ", 1)[1]
82.56 + classname = status.split(" ", 1)[1].replace(' ', '') # 304 Not Modified -> NotModified
82.57 + docstring = docstring or '`%s` status' % status
82.58 +
82.59 + def __init__(self, data=data, headers={}):
82.60 + HTTPError.__init__(self, status, headers, data)
82.61 +
82.62 + # trick to create class dynamically with dynamic docstring.
82.63 + return type(classname, (HTTPError, object), {
82.64 + '__doc__': docstring,
82.65 + '__init__': __init__
82.66 + })
82.67 +
82.68 +ok = OK = _status_code("200 OK", data="")
82.69 +created = Created = _status_code("201 Created")
82.70 +accepted = Accepted = _status_code("202 Accepted")
82.71 +
82.72 +class Redirect(HTTPError):
82.73 + """A `301 Moved Permanently` redirect."""
82.74 + def __init__(self, url, status='301 Moved Permanently', absolute=False):
82.75 + """
82.76 + Returns a `status` redirect to the new URL.
82.77 + `url` is joined with the base URL so that things like
82.78 + `redirect("about") will work properly.
82.79 + """
82.80 + newloc = urlparse.urljoin(ctx.path, url)
82.81 +
82.82 + if newloc.startswith('/'):
82.83 + if absolute:
82.84 + home = ctx.realhome
82.85 + else:
82.86 + home = ctx.home
82.87 + newloc = home + newloc
82.88 +
82.89 + headers = {
82.90 + 'Content-Type': 'text/html',
82.91 + 'Location': newloc
82.92 + }
82.93 + HTTPError.__init__(self, status, headers, "")
82.94 +
82.95 +redirect = Redirect
82.96 +
82.97 +class Found(Redirect):
82.98 + """A `302 Found` redirect."""
82.99 + def __init__(self, url, absolute=False):
82.100 + Redirect.__init__(self, url, '302 Found', absolute=absolute)
82.101 +
82.102 +found = Found
82.103 +
82.104 +class SeeOther(Redirect):
82.105 + """A `303 See Other` redirect."""
82.106 + def __init__(self, url, absolute=False):
82.107 + Redirect.__init__(self, url, '303 See Other', absolute=absolute)
82.108 +
82.109 +seeother = SeeOther
82.110 +
82.111 +class NotModified(HTTPError):
82.112 + """A `304 Not Modified` status."""
82.113 + def __init__(self):
82.114 + HTTPError.__init__(self, "304 Not Modified")
82.115 +
82.116 +notmodified = NotModified
82.117 +
82.118 +class TempRedirect(Redirect):
82.119 + """A `307 Temporary Redirect` redirect."""
82.120 + def __init__(self, url, absolute=False):
82.121 + Redirect.__init__(self, url, '307 Temporary Redirect', absolute=absolute)
82.122 +
82.123 +tempredirect = TempRedirect
82.124 +
82.125 +class BadRequest(HTTPError):
82.126 + """`400 Bad Request` error."""
82.127 + message = "bad request"
82.128 + def __init__(self, message=None):
82.129 + status = "400 Bad Request"
82.130 + headers = {'Content-Type': 'text/html'}
82.131 + HTTPError.__init__(self, status, headers, message or self.message)
82.132 +
82.133 +badrequest = BadRequest
82.134 +
82.135 +class Unauthorized(HTTPError):
82.136 + """`401 Unauthorized` error."""
82.137 + message = "unauthorized"
82.138 + def __init__(self):
82.139 + status = "401 Unauthorized"
82.140 + headers = {'Content-Type': 'text/html'}
82.141 + HTTPError.__init__(self, status, headers, self.message)
82.142 +
82.143 +unauthorized = Unauthorized
82.144 +
82.145 +class Forbidden(HTTPError):
82.146 + """`403 Forbidden` error."""
82.147 + message = "forbidden"
82.148 + def __init__(self):
82.149 + status = "403 Forbidden"
82.150 + headers = {'Content-Type': 'text/html'}
82.151 + HTTPError.__init__(self, status, headers, self.message)
82.152 +
82.153 +forbidden = Forbidden
82.154 +
82.155 +class _NotFound(HTTPError):
82.156 + """`404 Not Found` error."""
82.157 + message = "not found"
82.158 + def __init__(self, message=None):
82.159 + status = '404 Not Found'
82.160 + headers = {'Content-Type': 'text/html'}
82.161 + HTTPError.__init__(self, status, headers, message or self.message)
82.162 +
82.163 +def NotFound(message=None):
82.164 + """Returns HTTPError with '404 Not Found' error from the active application.
82.165 + """
82.166 + if message:
82.167 + return _NotFound(message)
82.168 + elif ctx.get('app_stack'):
82.169 + return ctx.app_stack[-1].notfound()
82.170 + else:
82.171 + return _NotFound()
82.172 +
82.173 +notfound = NotFound
82.174 +
82.175 +class NoMethod(HTTPError):
82.176 + """A `405 Method Not Allowed` error."""
82.177 + def __init__(self, cls=None):
82.178 + status = '405 Method Not Allowed'
82.179 + headers = {}
82.180 + headers['Content-Type'] = 'text/html'
82.181 +
82.182 + methods = ['GET', 'HEAD', 'POST', 'PUT', 'DELETE']
82.183 + if cls:
82.184 + methods = [method for method in methods if hasattr(cls, method)]
82.185 +
82.186 + headers['Allow'] = ', '.join(methods)
82.187 + data = None
82.188 + HTTPError.__init__(self, status, headers, data)
82.189 +
82.190 +nomethod = NoMethod
82.191 +
82.192 +class NotAcceptable(HTTPError):
82.193 + """`406 Not Acceptable` error."""
82.194 + message = "not acceptable"
82.195 + def __init__(self):
82.196 + status = "406 Not Acceptable"
82.197 + headers = {'Content-Type': 'text/html'}
82.198 + HTTPError.__init__(self, status, headers, self.message)
82.199 +
82.200 +notacceptable = NotAcceptable
82.201 +
82.202 +class Conflict(HTTPError):
82.203 + """`409 Conflict` error."""
82.204 + message = "conflict"
82.205 + def __init__(self):
82.206 + status = "409 Conflict"
82.207 + headers = {'Content-Type': 'text/html'}
82.208 + HTTPError.__init__(self, status, headers, self.message)
82.209 +
82.210 +conflict = Conflict
82.211 +
82.212 +class Gone(HTTPError):
82.213 + """`410 Gone` error."""
82.214 + message = "gone"
82.215 + def __init__(self):
82.216 + status = '410 Gone'
82.217 + headers = {'Content-Type': 'text/html'}
82.218 + HTTPError.__init__(self, status, headers, self.message)
82.219 +
82.220 +gone = Gone
82.221 +
82.222 +class PreconditionFailed(HTTPError):
82.223 + """`412 Precondition Failed` error."""
82.224 + message = "precondition failed"
82.225 + def __init__(self):
82.226 + status = "412 Precondition Failed"
82.227 + headers = {'Content-Type': 'text/html'}
82.228 + HTTPError.__init__(self, status, headers, self.message)
82.229 +
82.230 +preconditionfailed = PreconditionFailed
82.231 +
82.232 +class UnsupportedMediaType(HTTPError):
82.233 + """`415 Unsupported Media Type` error."""
82.234 + message = "unsupported media type"
82.235 + def __init__(self):
82.236 + status = "415 Unsupported Media Type"
82.237 + headers = {'Content-Type': 'text/html'}
82.238 + HTTPError.__init__(self, status, headers, self.message)
82.239 +
82.240 +unsupportedmediatype = UnsupportedMediaType
82.241 +
82.242 +class _InternalError(HTTPError):
82.243 + """500 Internal Server Error`."""
82.244 + message = "internal server error"
82.245 +
82.246 + def __init__(self, message=None):
82.247 + status = '500 Internal Server Error'
82.248 + headers = {'Content-Type': 'text/html'}
82.249 + HTTPError.__init__(self, status, headers, message or self.message)
82.250 +
82.251 +def InternalError(message=None):
82.252 + """Returns HTTPError with '500 internal error' error from the active application.
82.253 + """
82.254 + if message:
82.255 + return _InternalError(message)
82.256 + elif ctx.get('app_stack'):
82.257 + return ctx.app_stack[-1].internalerror()
82.258 + else:
82.259 + return _InternalError()
82.260 +
82.261 +internalerror = InternalError
82.262 +
82.263 +def header(hdr, value, unique=False):
82.264 + """
82.265 + Adds the header `hdr: value` with the response.
82.266 +
82.267 + If `unique` is True and a header with that name already exists,
82.268 + it doesn't add a new one.
82.269 + """
82.270 + hdr, value = safestr(hdr), safestr(value)
82.271 + # protection against HTTP response splitting attack
82.272 + if '\n' in hdr or '\r' in hdr or '\n' in value or '\r' in value:
82.273 + raise ValueError, 'invalid characters in header'
82.274 +
82.275 + if unique is True:
82.276 + for h, v in ctx.headers:
82.277 + if h.lower() == hdr.lower(): return
82.278 +
82.279 + ctx.headers.append((hdr, value))
82.280 +
82.281 +def rawinput(method=None):
82.282 + """Returns storage object with GET or POST arguments.
82.283 + """
82.284 + method = method or "both"
82.285 + from cStringIO import StringIO
82.286 +
82.287 + def dictify(fs):
82.288 + # hack to make web.input work with enctype='text/plain.
82.289 + if fs.list is None:
82.290 + fs.list = []
82.291 +
82.292 + return dict([(k, fs[k]) for k in fs.keys()])
82.293 +
82.294 + e = ctx.env.copy()
82.295 + a = b = {}
82.296 +
82.297 + if method.lower() in ['both', 'post', 'put']:
82.298 + if e['REQUEST_METHOD'] in ['POST', 'PUT']:
82.299 + if e.get('CONTENT_TYPE', '').lower().startswith('multipart/'):
82.300 + # since wsgi.input is directly passed to cgi.FieldStorage,
82.301 + # it can not be called multiple times. Saving the FieldStorage
82.302 + # object in ctx to allow calling web.input multiple times.
82.303 + a = ctx.get('_fieldstorage')
82.304 + if not a:
82.305 + fp = e['wsgi.input']
82.306 + a = cgi.FieldStorage(fp=fp, environ=e, keep_blank_values=1)
82.307 + ctx._fieldstorage = a
82.308 + else:
82.309 + fp = StringIO(data())
82.310 + a = cgi.FieldStorage(fp=fp, environ=e, keep_blank_values=1)
82.311 + a = dictify(a)
82.312 +
82.313 + if method.lower() in ['both', 'get']:
82.314 + e['REQUEST_METHOD'] = 'GET'
82.315 + b = dictify(cgi.FieldStorage(environ=e, keep_blank_values=1))
82.316 +
82.317 + def process_fieldstorage(fs):
82.318 + if isinstance(fs, list):
82.319 + return [process_fieldstorage(x) for x in fs]
82.320 + elif fs.filename is None:
82.321 + return fs.value
82.322 + else:
82.323 + return fs
82.324 +
82.325 + return storage([(k, process_fieldstorage(v)) for k, v in dictadd(b, a).items()])
82.326 +
82.327 +def input(*requireds, **defaults):
82.328 + """
82.329 + Returns a `storage` object with the GET and POST arguments.
82.330 + See `storify` for how `requireds` and `defaults` work.
82.331 + """
82.332 + _method = defaults.pop('_method', 'both')
82.333 + out = rawinput(_method)
82.334 + try:
82.335 + defaults.setdefault('_unicode', True) # force unicode conversion by default.
82.336 + return storify(out, *requireds, **defaults)
82.337 + except KeyError:
82.338 + raise badrequest()
82.339 +
82.340 +def data():
82.341 + """Returns the data sent with the request."""
82.342 + if 'data' not in ctx:
82.343 + cl = intget(ctx.env.get('CONTENT_LENGTH'), 0)
82.344 + ctx.data = ctx.env['wsgi.input'].read(cl)
82.345 + return ctx.data
82.346 +
82.347 +def setcookie(name, value, expires='', domain=None,
82.348 + secure=False, httponly=False, path=None):
82.349 + """Sets a cookie."""
82.350 + morsel = Cookie.Morsel()
82.351 + name, value = safestr(name), safestr(value)
82.352 + morsel.set(name, value, urllib.quote(value))
82.353 + if expires < 0:
82.354 + expires = -1000000000
82.355 + morsel['expires'] = expires
82.356 + morsel['path'] = path or ctx.homepath+'/'
82.357 + if domain:
82.358 + morsel['domain'] = domain
82.359 + if secure:
82.360 + morsel['secure'] = secure
82.361 + value = morsel.OutputString()
82.362 + if httponly:
82.363 + value += '; httponly'
82.364 + header('Set-Cookie', value)
82.365 +
82.366 +def decode_cookie(value):
82.367 + r"""Safely decodes a cookie value to unicode.
82.368 +
82.369 + Tries us-ascii, utf-8 and io8859 encodings, in that order.
82.370 +
82.371 + >>> decode_cookie('')
82.372 + u''
82.373 + >>> decode_cookie('asdf')
82.374 + u'asdf'
82.375 + >>> decode_cookie('foo \xC3\xA9 bar')
82.376 + u'foo \xe9 bar'
82.377 + >>> decode_cookie('foo \xE9 bar')
82.378 + u'foo \xe9 bar'
82.379 + """
82.380 + try:
82.381 + # First try plain ASCII encoding
82.382 + return unicode(value, 'us-ascii')
82.383 + except UnicodeError:
82.384 + # Then try UTF-8, and if that fails, ISO8859
82.385 + try:
82.386 + return unicode(value, 'utf-8')
82.387 + except UnicodeError:
82.388 + return unicode(value, 'iso8859', 'ignore')
82.389 +
82.390 +def parse_cookies(http_cookie):
82.391 + r"""Parse a HTTP_COOKIE header and return dict of cookie names and decoded values.
82.392 +
82.393 + >>> sorted(parse_cookies('').items())
82.394 + []
82.395 + >>> sorted(parse_cookies('a=1').items())
82.396 + [('a', '1')]
82.397 + >>> sorted(parse_cookies('a=1%202').items())
82.398 + [('a', '1 2')]
82.399 + >>> sorted(parse_cookies('a=Z%C3%A9Z').items())
82.400 + [('a', 'Z\xc3\xa9Z')]
82.401 + >>> sorted(parse_cookies('a=1; b=2; c=3').items())
82.402 + [('a', '1'), ('b', '2'), ('c', '3')]
82.403 + >>> sorted(parse_cookies('a=1; b=w("x")|y=z; c=3').items())
82.404 + [('a', '1'), ('b', 'w('), ('c', '3')]
82.405 + >>> sorted(parse_cookies('a=1; b=w(%22x%22)|y=z; c=3').items())
82.406 + [('a', '1'), ('b', 'w("x")|y=z'), ('c', '3')]
82.407 +
82.408 + >>> sorted(parse_cookies('keebler=E=mc2').items())
82.409 + [('keebler', 'E=mc2')]
82.410 + >>> sorted(parse_cookies(r'keebler="E=mc2; L=\"Loves\"; fudge=\012;"').items())
82.411 + [('keebler', 'E=mc2; L="Loves"; fudge=\n;')]
82.412 + """
82.413 + #print "parse_cookies"
82.414 + if '"' in http_cookie:
82.415 + # HTTP_COOKIE has quotes in it, use slow but correct cookie parsing
82.416 + cookie = Cookie.SimpleCookie()
82.417 + try:
82.418 + cookie.load(http_cookie)
82.419 + except Cookie.CookieError:
82.420 + # If HTTP_COOKIE header is malformed, try at least to load the cookies we can by
82.421 + # first splitting on ';' and loading each attr=value pair separately
82.422 + cookie = Cookie.SimpleCookie()
82.423 + for attr_value in http_cookie.split(';'):
82.424 + try:
82.425 + cookie.load(attr_value)
82.426 + except Cookie.CookieError:
82.427 + pass
82.428 + cookies = dict((k, urllib.unquote(v.value)) for k, v in cookie.iteritems())
82.429 + else:
82.430 + # HTTP_COOKIE doesn't have quotes, use fast cookie parsing
82.431 + cookies = {}
82.432 + for key_value in http_cookie.split(';'):
82.433 + key_value = key_value.split('=', 1)
82.434 + if len(key_value) == 2:
82.435 + key, value = key_value
82.436 + cookies[key.strip()] = urllib.unquote(value.strip())
82.437 + return cookies
82.438 +
82.439 +def cookies(*requireds, **defaults):
82.440 + r"""Returns a `storage` object with all the request cookies in it.
82.441 +
82.442 + See `storify` for how `requireds` and `defaults` work.
82.443 +
82.444 + This is forgiving on bad HTTP_COOKIE input, it tries to parse at least
82.445 + the cookies it can.
82.446 +
82.447 + The values are converted to unicode if _unicode=True is passed.
82.448 + """
82.449 + # If _unicode=True is specified, use decode_cookie to convert cookie value to unicode
82.450 + if defaults.get("_unicode") is True:
82.451 + defaults['_unicode'] = decode_cookie
82.452 +
82.453 + # parse cookie string and cache the result for next time.
82.454 + if '_parsed_cookies' not in ctx:
82.455 + http_cookie = ctx.env.get("HTTP_COOKIE", "")
82.456 + ctx._parsed_cookies = parse_cookies(http_cookie)
82.457 +
82.458 + try:
82.459 + return storify(ctx._parsed_cookies, *requireds, **defaults)
82.460 + except KeyError:
82.461 + badrequest()
82.462 + raise StopIteration
82.463 +
82.464 +def debug(*args):
82.465 + """
82.466 + Prints a prettyprinted version of `args` to stderr.
82.467 + """
82.468 + try:
82.469 + out = ctx.environ['wsgi.errors']
82.470 + except:
82.471 + out = sys.stderr
82.472 + for arg in args:
82.473 + print >> out, pprint.pformat(arg)
82.474 + return ''
82.475 +
82.476 +def _debugwrite(x):
82.477 + try:
82.478 + out = ctx.environ['wsgi.errors']
82.479 + except:
82.480 + out = sys.stderr
82.481 + out.write(x)
82.482 +debug.write = _debugwrite
82.483 +
82.484 +ctx = context = threadeddict()
82.485 +
82.486 +ctx.__doc__ = """
82.487 +A `storage` object containing various information about the request:
82.488 +
82.489 +`environ` (aka `env`)
82.490 + : A dictionary containing the standard WSGI environment variables.
82.491 +
82.492 +`host`
82.493 + : The domain (`Host` header) requested by the user.
82.494 +
82.495 +`home`
82.496 + : The base path for the application.
82.497 +
82.498 +`ip`
82.499 + : The IP address of the requester.
82.500 +
82.501 +`method`
82.502 + : The HTTP method used.
82.503 +
82.504 +`path`
82.505 + : The path request.
82.506 +
82.507 +`query`
82.508 + : If there are no query arguments, the empty string. Otherwise, a `?` followed
82.509 + by the query string.
82.510 +
82.511 +`fullpath`
82.512 + : The full path requested, including query arguments (`== path + query`).
82.513 +
82.514 +### Response Data
82.515 +
82.516 +`status` (default: "200 OK")
82.517 + : The status code to be used in the response.
82.518 +
82.519 +`headers`
82.520 + : A list of 2-tuples to be used in the response.
82.521 +
82.522 +`output`
82.523 + : A string to be used as the response.
82.524 +"""
82.525 +
82.526 +if __name__ == "__main__":
82.527 + import doctest
82.528 + doctest.testmod()
82.529 \ No newline at end of file
83.1 Binary file OpenSecurity/install/web.py-0.37/web/webapi.pyc has changed
84.1 --- /dev/null Thu Jan 01 00:00:00 1970 +0000
84.2 +++ b/OpenSecurity/install/web.py-0.37/web/webopenid.py Mon Dec 02 14:02:05 2013 +0100
84.3 @@ -0,0 +1,115 @@
84.4 +"""openid.py: an openid library for web.py
84.5 +
84.6 +Notes:
84.7 +
84.8 + - This will create a file called .openid_secret_key in the
84.9 + current directory with your secret key in it. If someone
84.10 + has access to this file they can log in as any user. And
84.11 + if the app can't find this file for any reason (e.g. you
84.12 + moved the app somewhere else) then each currently logged
84.13 + in user will get logged out.
84.14 +
84.15 + - State must be maintained through the entire auth process
84.16 + -- this means that if you have multiple web.py processes
84.17 + serving one set of URLs or if you restart your app often
84.18 + then log ins will fail. You have to replace sessions and
84.19 + store for things to work.
84.20 +
84.21 + - We set cookies starting with "openid_".
84.22 +
84.23 +"""
84.24 +
84.25 +import os
84.26 +import random
84.27 +import hmac
84.28 +import __init__ as web
84.29 +import openid.consumer.consumer
84.30 +import openid.store.memstore
84.31 +
84.32 +sessions = {}
84.33 +store = openid.store.memstore.MemoryStore()
84.34 +
84.35 +def _secret():
84.36 + try:
84.37 + secret = file('.openid_secret_key').read()
84.38 + except IOError:
84.39 + # file doesn't exist
84.40 + secret = os.urandom(20)
84.41 + file('.openid_secret_key', 'w').write(secret)
84.42 + return secret
84.43 +
84.44 +def _hmac(identity_url):
84.45 + return hmac.new(_secret(), identity_url).hexdigest()
84.46 +
84.47 +def _random_session():
84.48 + n = random.random()
84.49 + while n in sessions:
84.50 + n = random.random()
84.51 + n = str(n)
84.52 + return n
84.53 +
84.54 +def status():
84.55 + oid_hash = web.cookies().get('openid_identity_hash', '').split(',', 1)
84.56 + if len(oid_hash) > 1:
84.57 + oid_hash, identity_url = oid_hash
84.58 + if oid_hash == _hmac(identity_url):
84.59 + return identity_url
84.60 + return None
84.61 +
84.62 +def form(openid_loc):
84.63 + oid = status()
84.64 + if oid:
84.65 + return '''
84.66 + <form method="post" action="%s">
84.67 + <img src="http://openid.net/login-bg.gif" alt="OpenID" />
84.68 + <strong>%s</strong>
84.69 + <input type="hidden" name="action" value="logout" />
84.70 + <input type="hidden" name="return_to" value="%s" />
84.71 + <button type="submit">log out</button>
84.72 + </form>''' % (openid_loc, oid, web.ctx.fullpath)
84.73 + else:
84.74 + return '''
84.75 + <form method="post" action="%s">
84.76 + <input type="text" name="openid" value=""
84.77 + style="background: url(http://openid.net/login-bg.gif) no-repeat; padding-left: 18px; background-position: 0 50%%;" />
84.78 + <input type="hidden" name="return_to" value="%s" />
84.79 + <button type="submit">log in</button>
84.80 + </form>''' % (openid_loc, web.ctx.fullpath)
84.81 +
84.82 +def logout():
84.83 + web.setcookie('openid_identity_hash', '', expires=-1)
84.84 +
84.85 +class host:
84.86 + def POST(self):
84.87 + # unlike the usual scheme of things, the POST is actually called
84.88 + # first here
84.89 + i = web.input(return_to='/')
84.90 + if i.get('action') == 'logout':
84.91 + logout()
84.92 + return web.redirect(i.return_to)
84.93 +
84.94 + i = web.input('openid', return_to='/')
84.95 +
84.96 + n = _random_session()
84.97 + sessions[n] = {'webpy_return_to': i.return_to}
84.98 +
84.99 + c = openid.consumer.consumer.Consumer(sessions[n], store)
84.100 + a = c.begin(i.openid)
84.101 + f = a.redirectURL(web.ctx.home, web.ctx.home + web.ctx.fullpath)
84.102 +
84.103 + web.setcookie('openid_session_id', n)
84.104 + return web.redirect(f)
84.105 +
84.106 + def GET(self):
84.107 + n = web.cookies('openid_session_id').openid_session_id
84.108 + web.setcookie('openid_session_id', '', expires=-1)
84.109 + return_to = sessions[n]['webpy_return_to']
84.110 +
84.111 + c = openid.consumer.consumer.Consumer(sessions[n], store)
84.112 + a = c.complete(web.input(), web.ctx.home + web.ctx.fullpath)
84.113 +
84.114 + if a.status.lower() == 'success':
84.115 + web.setcookie('openid_identity_hash', _hmac(a.identity_url) + ',' + a.identity_url)
84.116 +
84.117 + del sessions[n]
84.118 + return web.redirect(return_to)
85.1 Binary file OpenSecurity/install/web.py-0.37/web/webopenid.pyc has changed
86.1 --- /dev/null Thu Jan 01 00:00:00 1970 +0000
86.2 +++ b/OpenSecurity/install/web.py-0.37/web/wsgi.py Mon Dec 02 14:02:05 2013 +0100
86.3 @@ -0,0 +1,70 @@
86.4 +"""
86.5 +WSGI Utilities
86.6 +(from web.py)
86.7 +"""
86.8 +
86.9 +import os, sys
86.10 +
86.11 +import http
86.12 +import webapi as web
86.13 +from utils import listget
86.14 +from net import validaddr, validip
86.15 +import httpserver
86.16 +
86.17 +def runfcgi(func, addr=('localhost', 8000)):
86.18 + """Runs a WSGI function as a FastCGI server."""
86.19 + import flup.server.fcgi as flups
86.20 + return flups.WSGIServer(func, multiplexed=True, bindAddress=addr, debug=False).run()
86.21 +
86.22 +def runscgi(func, addr=('localhost', 4000)):
86.23 + """Runs a WSGI function as an SCGI server."""
86.24 + import flup.server.scgi as flups
86.25 + return flups.WSGIServer(func, bindAddress=addr, debug=False).run()
86.26 +
86.27 +def runwsgi(func):
86.28 + """
86.29 + Runs a WSGI-compatible `func` using FCGI, SCGI, or a simple web server,
86.30 + as appropriate based on context and `sys.argv`.
86.31 + """
86.32 +
86.33 + if os.environ.has_key('SERVER_SOFTWARE'): # cgi
86.34 + os.environ['FCGI_FORCE_CGI'] = 'Y'
86.35 +
86.36 + if (os.environ.has_key('PHP_FCGI_CHILDREN') #lighttpd fastcgi
86.37 + or os.environ.has_key('SERVER_SOFTWARE')):
86.38 + return runfcgi(func, None)
86.39 +
86.40 + if 'fcgi' in sys.argv or 'fastcgi' in sys.argv:
86.41 + args = sys.argv[1:]
86.42 + if 'fastcgi' in args: args.remove('fastcgi')
86.43 + elif 'fcgi' in args: args.remove('fcgi')
86.44 + if args:
86.45 + return runfcgi(func, validaddr(args[0]))
86.46 + else:
86.47 + return runfcgi(func, None)
86.48 +
86.49 + if 'scgi' in sys.argv:
86.50 + args = sys.argv[1:]
86.51 + args.remove('scgi')
86.52 + if args:
86.53 + return runscgi(func, validaddr(args[0]))
86.54 + else:
86.55 + return runscgi(func)
86.56 +
86.57 + return httpserver.runsimple(func, validip(listget(sys.argv, 1, '')))
86.58 +
86.59 +def _is_dev_mode():
86.60 + # Some embedded python interpreters won't have sys.arv
86.61 + # For details, see https://github.com/webpy/webpy/issues/87
86.62 + argv = getattr(sys, "argv", [])
86.63 +
86.64 + # quick hack to check if the program is running in dev mode.
86.65 + if os.environ.has_key('SERVER_SOFTWARE') \
86.66 + or os.environ.has_key('PHP_FCGI_CHILDREN') \
86.67 + or 'fcgi' in argv or 'fastcgi' in argv \
86.68 + or 'mod_wsgi' in argv:
86.69 + return False
86.70 + return True
86.71 +
86.72 +# When running the builtin-server, enable debug mode if not already set.
86.73 +web.config.setdefault('debug', _is_dev_mode())
87.1 Binary file OpenSecurity/install/web.py-0.37/web/wsgi.pyc has changed
88.1 --- /dev/null Thu Jan 01 00:00:00 1970 +0000
88.2 +++ b/OpenSecurity/install/web.py-0.37/web/wsgiserver/__init__.py Mon Dec 02 14:02:05 2013 +0100
88.3 @@ -0,0 +1,2219 @@
88.4 +"""A high-speed, production ready, thread pooled, generic HTTP server.
88.5 +
88.6 +Simplest example on how to use this module directly
88.7 +(without using CherryPy's application machinery)::
88.8 +
88.9 + from cherrypy import wsgiserver
88.10 +
88.11 + def my_crazy_app(environ, start_response):
88.12 + status = '200 OK'
88.13 + response_headers = [('Content-type','text/plain')]
88.14 + start_response(status, response_headers)
88.15 + return ['Hello world!']
88.16 +
88.17 + server = wsgiserver.CherryPyWSGIServer(
88.18 + ('0.0.0.0', 8070), my_crazy_app,
88.19 + server_name='www.cherrypy.example')
88.20 + server.start()
88.21 +
88.22 +The CherryPy WSGI server can serve as many WSGI applications
88.23 +as you want in one instance by using a WSGIPathInfoDispatcher::
88.24 +
88.25 + d = WSGIPathInfoDispatcher({'/': my_crazy_app, '/blog': my_blog_app})
88.26 + server = wsgiserver.CherryPyWSGIServer(('0.0.0.0', 80), d)
88.27 +
88.28 +Want SSL support? Just set server.ssl_adapter to an SSLAdapter instance.
88.29 +
88.30 +This won't call the CherryPy engine (application side) at all, only the
88.31 +HTTP server, which is independent from the rest of CherryPy. Don't
88.32 +let the name "CherryPyWSGIServer" throw you; the name merely reflects
88.33 +its origin, not its coupling.
88.34 +
88.35 +For those of you wanting to understand internals of this module, here's the
88.36 +basic call flow. The server's listening thread runs a very tight loop,
88.37 +sticking incoming connections onto a Queue::
88.38 +
88.39 + server = CherryPyWSGIServer(...)
88.40 + server.start()
88.41 + while True:
88.42 + tick()
88.43 + # This blocks until a request comes in:
88.44 + child = socket.accept()
88.45 + conn = HTTPConnection(child, ...)
88.46 + server.requests.put(conn)
88.47 +
88.48 +Worker threads are kept in a pool and poll the Queue, popping off and then
88.49 +handling each connection in turn. Each connection can consist of an arbitrary
88.50 +number of requests and their responses, so we run a nested loop::
88.51 +
88.52 + while True:
88.53 + conn = server.requests.get()
88.54 + conn.communicate()
88.55 + -> while True:
88.56 + req = HTTPRequest(...)
88.57 + req.parse_request()
88.58 + -> # Read the Request-Line, e.g. "GET /page HTTP/1.1"
88.59 + req.rfile.readline()
88.60 + read_headers(req.rfile, req.inheaders)
88.61 + req.respond()
88.62 + -> response = app(...)
88.63 + try:
88.64 + for chunk in response:
88.65 + if chunk:
88.66 + req.write(chunk)
88.67 + finally:
88.68 + if hasattr(response, "close"):
88.69 + response.close()
88.70 + if req.close_connection:
88.71 + return
88.72 +"""
88.73 +
88.74 +CRLF = '\r\n'
88.75 +import os
88.76 +import Queue
88.77 +import re
88.78 +quoted_slash = re.compile("(?i)%2F")
88.79 +import rfc822
88.80 +import socket
88.81 +import sys
88.82 +if 'win' in sys.platform and not hasattr(socket, 'IPPROTO_IPV6'):
88.83 + socket.IPPROTO_IPV6 = 41
88.84 +try:
88.85 + import cStringIO as StringIO
88.86 +except ImportError:
88.87 + import StringIO
88.88 +DEFAULT_BUFFER_SIZE = -1
88.89 +
88.90 +_fileobject_uses_str_type = isinstance(socket._fileobject(None)._rbuf, basestring)
88.91 +
88.92 +import threading
88.93 +import time
88.94 +import traceback
88.95 +def format_exc(limit=None):
88.96 + """Like print_exc() but return a string. Backport for Python 2.3."""
88.97 + try:
88.98 + etype, value, tb = sys.exc_info()
88.99 + return ''.join(traceback.format_exception(etype, value, tb, limit))
88.100 + finally:
88.101 + etype = value = tb = None
88.102 +
88.103 +
88.104 +from urllib import unquote
88.105 +from urlparse import urlparse
88.106 +import warnings
88.107 +
88.108 +import errno
88.109 +
88.110 +def plat_specific_errors(*errnames):
88.111 + """Return error numbers for all errors in errnames on this platform.
88.112 +
88.113 + The 'errno' module contains different global constants depending on
88.114 + the specific platform (OS). This function will return the list of
88.115 + numeric values for a given list of potential names.
88.116 + """
88.117 + errno_names = dir(errno)
88.118 + nums = [getattr(errno, k) for k in errnames if k in errno_names]
88.119 + # de-dupe the list
88.120 + return dict.fromkeys(nums).keys()
88.121 +
88.122 +socket_error_eintr = plat_specific_errors("EINTR", "WSAEINTR")
88.123 +
88.124 +socket_errors_to_ignore = plat_specific_errors(
88.125 + "EPIPE",
88.126 + "EBADF", "WSAEBADF",
88.127 + "ENOTSOCK", "WSAENOTSOCK",
88.128 + "ETIMEDOUT", "WSAETIMEDOUT",
88.129 + "ECONNREFUSED", "WSAECONNREFUSED",
88.130 + "ECONNRESET", "WSAECONNRESET",
88.131 + "ECONNABORTED", "WSAECONNABORTED",
88.132 + "ENETRESET", "WSAENETRESET",
88.133 + "EHOSTDOWN", "EHOSTUNREACH",
88.134 + )
88.135 +socket_errors_to_ignore.append("timed out")
88.136 +socket_errors_to_ignore.append("The read operation timed out")
88.137 +
88.138 +socket_errors_nonblocking = plat_specific_errors(
88.139 + 'EAGAIN', 'EWOULDBLOCK', 'WSAEWOULDBLOCK')
88.140 +
88.141 +comma_separated_headers = ['Accept', 'Accept-Charset', 'Accept-Encoding',
88.142 + 'Accept-Language', 'Accept-Ranges', 'Allow', 'Cache-Control',
88.143 + 'Connection', 'Content-Encoding', 'Content-Language', 'Expect',
88.144 + 'If-Match', 'If-None-Match', 'Pragma', 'Proxy-Authenticate', 'TE',
88.145 + 'Trailer', 'Transfer-Encoding', 'Upgrade', 'Vary', 'Via', 'Warning',
88.146 + 'WWW-Authenticate']
88.147 +
88.148 +
88.149 +import logging
88.150 +if not hasattr(logging, 'statistics'): logging.statistics = {}
88.151 +
88.152 +
88.153 +def read_headers(rfile, hdict=None):
88.154 + """Read headers from the given stream into the given header dict.
88.155 +
88.156 + If hdict is None, a new header dict is created. Returns the populated
88.157 + header dict.
88.158 +
88.159 + Headers which are repeated are folded together using a comma if their
88.160 + specification so dictates.
88.161 +
88.162 + This function raises ValueError when the read bytes violate the HTTP spec.
88.163 + You should probably return "400 Bad Request" if this happens.
88.164 + """
88.165 + if hdict is None:
88.166 + hdict = {}
88.167 +
88.168 + while True:
88.169 + line = rfile.readline()
88.170 + if not line:
88.171 + # No more data--illegal end of headers
88.172 + raise ValueError("Illegal end of headers.")
88.173 +
88.174 + if line == CRLF:
88.175 + # Normal end of headers
88.176 + break
88.177 + if not line.endswith(CRLF):
88.178 + raise ValueError("HTTP requires CRLF terminators")
88.179 +
88.180 + if line[0] in ' \t':
88.181 + # It's a continuation line.
88.182 + v = line.strip()
88.183 + else:
88.184 + try:
88.185 + k, v = line.split(":", 1)
88.186 + except ValueError:
88.187 + raise ValueError("Illegal header line.")
88.188 + # TODO: what about TE and WWW-Authenticate?
88.189 + k = k.strip().title()
88.190 + v = v.strip()
88.191 + hname = k
88.192 +
88.193 + if k in comma_separated_headers:
88.194 + existing = hdict.get(hname)
88.195 + if existing:
88.196 + v = ", ".join((existing, v))
88.197 + hdict[hname] = v
88.198 +
88.199 + return hdict
88.200 +
88.201 +
88.202 +class MaxSizeExceeded(Exception):
88.203 + pass
88.204 +
88.205 +class SizeCheckWrapper(object):
88.206 + """Wraps a file-like object, raising MaxSizeExceeded if too large."""
88.207 +
88.208 + def __init__(self, rfile, maxlen):
88.209 + self.rfile = rfile
88.210 + self.maxlen = maxlen
88.211 + self.bytes_read = 0
88.212 +
88.213 + def _check_length(self):
88.214 + if self.maxlen and self.bytes_read > self.maxlen:
88.215 + raise MaxSizeExceeded()
88.216 +
88.217 + def read(self, size=None):
88.218 + data = self.rfile.read(size)
88.219 + self.bytes_read += len(data)
88.220 + self._check_length()
88.221 + return data
88.222 +
88.223 + def readline(self, size=None):
88.224 + if size is not None:
88.225 + data = self.rfile.readline(size)
88.226 + self.bytes_read += len(data)
88.227 + self._check_length()
88.228 + return data
88.229 +
88.230 + # User didn't specify a size ...
88.231 + # We read the line in chunks to make sure it's not a 100MB line !
88.232 + res = []
88.233 + while True:
88.234 + data = self.rfile.readline(256)
88.235 + self.bytes_read += len(data)
88.236 + self._check_length()
88.237 + res.append(data)
88.238 + # See http://www.cherrypy.org/ticket/421
88.239 + if len(data) < 256 or data[-1:] == "\n":
88.240 + return ''.join(res)
88.241 +
88.242 + def readlines(self, sizehint=0):
88.243 + # Shamelessly stolen from StringIO
88.244 + total = 0
88.245 + lines = []
88.246 + line = self.readline()
88.247 + while line:
88.248 + lines.append(line)
88.249 + total += len(line)
88.250 + if 0 < sizehint <= total:
88.251 + break
88.252 + line = self.readline()
88.253 + return lines
88.254 +
88.255 + def close(self):
88.256 + self.rfile.close()
88.257 +
88.258 + def __iter__(self):
88.259 + return self
88.260 +
88.261 + def next(self):
88.262 + data = self.rfile.next()
88.263 + self.bytes_read += len(data)
88.264 + self._check_length()
88.265 + return data
88.266 +
88.267 +
88.268 +class KnownLengthRFile(object):
88.269 + """Wraps a file-like object, returning an empty string when exhausted."""
88.270 +
88.271 + def __init__(self, rfile, content_length):
88.272 + self.rfile = rfile
88.273 + self.remaining = content_length
88.274 +
88.275 + def read(self, size=None):
88.276 + if self.remaining == 0:
88.277 + return ''
88.278 + if size is None:
88.279 + size = self.remaining
88.280 + else:
88.281 + size = min(size, self.remaining)
88.282 +
88.283 + data = self.rfile.read(size)
88.284 + self.remaining -= len(data)
88.285 + return data
88.286 +
88.287 + def readline(self, size=None):
88.288 + if self.remaining == 0:
88.289 + return ''
88.290 + if size is None:
88.291 + size = self.remaining
88.292 + else:
88.293 + size = min(size, self.remaining)
88.294 +
88.295 + data = self.rfile.readline(size)
88.296 + self.remaining -= len(data)
88.297 + return data
88.298 +
88.299 + def readlines(self, sizehint=0):
88.300 + # Shamelessly stolen from StringIO
88.301 + total = 0
88.302 + lines = []
88.303 + line = self.readline(sizehint)
88.304 + while line:
88.305 + lines.append(line)
88.306 + total += len(line)
88.307 + if 0 < sizehint <= total:
88.308 + break
88.309 + line = self.readline(sizehint)
88.310 + return lines
88.311 +
88.312 + def close(self):
88.313 + self.rfile.close()
88.314 +
88.315 + def __iter__(self):
88.316 + return self
88.317 +
88.318 + def __next__(self):
88.319 + data = next(self.rfile)
88.320 + self.remaining -= len(data)
88.321 + return data
88.322 +
88.323 +
88.324 +class ChunkedRFile(object):
88.325 + """Wraps a file-like object, returning an empty string when exhausted.
88.326 +
88.327 + This class is intended to provide a conforming wsgi.input value for
88.328 + request entities that have been encoded with the 'chunked' transfer
88.329 + encoding.
88.330 + """
88.331 +
88.332 + def __init__(self, rfile, maxlen, bufsize=8192):
88.333 + self.rfile = rfile
88.334 + self.maxlen = maxlen
88.335 + self.bytes_read = 0
88.336 + self.buffer = ''
88.337 + self.bufsize = bufsize
88.338 + self.closed = False
88.339 +
88.340 + def _fetch(self):
88.341 + if self.closed:
88.342 + return
88.343 +
88.344 + line = self.rfile.readline()
88.345 + self.bytes_read += len(line)
88.346 +
88.347 + if self.maxlen and self.bytes_read > self.maxlen:
88.348 + raise MaxSizeExceeded("Request Entity Too Large", self.maxlen)
88.349 +
88.350 + line = line.strip().split(";", 1)
88.351 +
88.352 + try:
88.353 + chunk_size = line.pop(0)
88.354 + chunk_size = int(chunk_size, 16)
88.355 + except ValueError:
88.356 + raise ValueError("Bad chunked transfer size: " + repr(chunk_size))
88.357 +
88.358 + if chunk_size <= 0:
88.359 + self.closed = True
88.360 + return
88.361 +
88.362 +## if line: chunk_extension = line[0]
88.363 +
88.364 + if self.maxlen and self.bytes_read + chunk_size > self.maxlen:
88.365 + raise IOError("Request Entity Too Large")
88.366 +
88.367 + chunk = self.rfile.read(chunk_size)
88.368 + self.bytes_read += len(chunk)
88.369 + self.buffer += chunk
88.370 +
88.371 + crlf = self.rfile.read(2)
88.372 + if crlf != CRLF:
88.373 + raise ValueError(
88.374 + "Bad chunked transfer coding (expected '\\r\\n', "
88.375 + "got " + repr(crlf) + ")")
88.376 +
88.377 + def read(self, size=None):
88.378 + data = ''
88.379 + while True:
88.380 + if size and len(data) >= size:
88.381 + return data
88.382 +
88.383 + if not self.buffer:
88.384 + self._fetch()
88.385 + if not self.buffer:
88.386 + # EOF
88.387 + return data
88.388 +
88.389 + if size:
88.390 + remaining = size - len(data)
88.391 + data += self.buffer[:remaining]
88.392 + self.buffer = self.buffer[remaining:]
88.393 + else:
88.394 + data += self.buffer
88.395 +
88.396 + def readline(self, size=None):
88.397 + data = ''
88.398 + while True:
88.399 + if size and len(data) >= size:
88.400 + return data
88.401 +
88.402 + if not self.buffer:
88.403 + self._fetch()
88.404 + if not self.buffer:
88.405 + # EOF
88.406 + return data
88.407 +
88.408 + newline_pos = self.buffer.find('\n')
88.409 + if size:
88.410 + if newline_pos == -1:
88.411 + remaining = size - len(data)
88.412 + data += self.buffer[:remaining]
88.413 + self.buffer = self.buffer[remaining:]
88.414 + else:
88.415 + remaining = min(size - len(data), newline_pos)
88.416 + data += self.buffer[:remaining]
88.417 + self.buffer = self.buffer[remaining:]
88.418 + else:
88.419 + if newline_pos == -1:
88.420 + data += self.buffer
88.421 + else:
88.422 + data += self.buffer[:newline_pos]
88.423 + self.buffer = self.buffer[newline_pos:]
88.424 +
88.425 + def readlines(self, sizehint=0):
88.426 + # Shamelessly stolen from StringIO
88.427 + total = 0
88.428 + lines = []
88.429 + line = self.readline(sizehint)
88.430 + while line:
88.431 + lines.append(line)
88.432 + total += len(line)
88.433 + if 0 < sizehint <= total:
88.434 + break
88.435 + line = self.readline(sizehint)
88.436 + return lines
88.437 +
88.438 + def read_trailer_lines(self):
88.439 + if not self.closed:
88.440 + raise ValueError(
88.441 + "Cannot read trailers until the request body has been read.")
88.442 +
88.443 + while True:
88.444 + line = self.rfile.readline()
88.445 + if not line:
88.446 + # No more data--illegal end of headers
88.447 + raise ValueError("Illegal end of headers.")
88.448 +
88.449 + self.bytes_read += len(line)
88.450 + if self.maxlen and self.bytes_read > self.maxlen:
88.451 + raise IOError("Request Entity Too Large")
88.452 +
88.453 + if line == CRLF:
88.454 + # Normal end of headers
88.455 + break
88.456 + if not line.endswith(CRLF):
88.457 + raise ValueError("HTTP requires CRLF terminators")
88.458 +
88.459 + yield line
88.460 +
88.461 + def close(self):
88.462 + self.rfile.close()
88.463 +
88.464 + def __iter__(self):
88.465 + # Shamelessly stolen from StringIO
88.466 + total = 0
88.467 + line = self.readline(sizehint)
88.468 + while line:
88.469 + yield line
88.470 + total += len(line)
88.471 + if 0 < sizehint <= total:
88.472 + break
88.473 + line = self.readline(sizehint)
88.474 +
88.475 +
88.476 +class HTTPRequest(object):
88.477 + """An HTTP Request (and response).
88.478 +
88.479 + A single HTTP connection may consist of multiple request/response pairs.
88.480 + """
88.481 +
88.482 + server = None
88.483 + """The HTTPServer object which is receiving this request."""
88.484 +
88.485 + conn = None
88.486 + """The HTTPConnection object on which this request connected."""
88.487 +
88.488 + inheaders = {}
88.489 + """A dict of request headers."""
88.490 +
88.491 + outheaders = []
88.492 + """A list of header tuples to write in the response."""
88.493 +
88.494 + ready = False
88.495 + """When True, the request has been parsed and is ready to begin generating
88.496 + the response. When False, signals the calling Connection that the response
88.497 + should not be generated and the connection should close."""
88.498 +
88.499 + close_connection = False
88.500 + """Signals the calling Connection that the request should close. This does
88.501 + not imply an error! The client and/or server may each request that the
88.502 + connection be closed."""
88.503 +
88.504 + chunked_write = False
88.505 + """If True, output will be encoded with the "chunked" transfer-coding.
88.506 +
88.507 + This value is set automatically inside send_headers."""
88.508 +
88.509 + def __init__(self, server, conn):
88.510 + self.server= server
88.511 + self.conn = conn
88.512 +
88.513 + self.ready = False
88.514 + self.started_request = False
88.515 + self.scheme = "http"
88.516 + if self.server.ssl_adapter is not None:
88.517 + self.scheme = "https"
88.518 + # Use the lowest-common protocol in case read_request_line errors.
88.519 + self.response_protocol = 'HTTP/1.0'
88.520 + self.inheaders = {}
88.521 +
88.522 + self.status = ""
88.523 + self.outheaders = []
88.524 + self.sent_headers = False
88.525 + self.close_connection = self.__class__.close_connection
88.526 + self.chunked_read = False
88.527 + self.chunked_write = self.__class__.chunked_write
88.528 +
88.529 + def parse_request(self):
88.530 + """Parse the next HTTP request start-line and message-headers."""
88.531 + self.rfile = SizeCheckWrapper(self.conn.rfile,
88.532 + self.server.max_request_header_size)
88.533 + try:
88.534 + self.read_request_line()
88.535 + except MaxSizeExceeded:
88.536 + self.simple_response("414 Request-URI Too Long",
88.537 + "The Request-URI sent with the request exceeds the maximum "
88.538 + "allowed bytes.")
88.539 + return
88.540 +
88.541 + try:
88.542 + success = self.read_request_headers()
88.543 + except MaxSizeExceeded:
88.544 + self.simple_response("413 Request Entity Too Large",
88.545 + "The headers sent with the request exceed the maximum "
88.546 + "allowed bytes.")
88.547 + return
88.548 + else:
88.549 + if not success:
88.550 + return
88.551 +
88.552 + self.ready = True
88.553 +
88.554 + def read_request_line(self):
88.555 + # HTTP/1.1 connections are persistent by default. If a client
88.556 + # requests a page, then idles (leaves the connection open),
88.557 + # then rfile.readline() will raise socket.error("timed out").
88.558 + # Note that it does this based on the value given to settimeout(),
88.559 + # and doesn't need the client to request or acknowledge the close
88.560 + # (although your TCP stack might suffer for it: cf Apache's history
88.561 + # with FIN_WAIT_2).
88.562 + request_line = self.rfile.readline()
88.563 +
88.564 + # Set started_request to True so communicate() knows to send 408
88.565 + # from here on out.
88.566 + self.started_request = True
88.567 + if not request_line:
88.568 + # Force self.ready = False so the connection will close.
88.569 + self.ready = False
88.570 + return
88.571 +
88.572 + if request_line == CRLF:
88.573 + # RFC 2616 sec 4.1: "...if the server is reading the protocol
88.574 + # stream at the beginning of a message and receives a CRLF
88.575 + # first, it should ignore the CRLF."
88.576 + # But only ignore one leading line! else we enable a DoS.
88.577 + request_line = self.rfile.readline()
88.578 + if not request_line:
88.579 + self.ready = False
88.580 + return
88.581 +
88.582 + if not request_line.endswith(CRLF):
88.583 + self.simple_response("400 Bad Request", "HTTP requires CRLF terminators")
88.584 + return
88.585 +
88.586 + try:
88.587 + method, uri, req_protocol = request_line.strip().split(" ", 2)
88.588 + rp = int(req_protocol[5]), int(req_protocol[7])
88.589 + except (ValueError, IndexError):
88.590 + self.simple_response("400 Bad Request", "Malformed Request-Line")
88.591 + return
88.592 +
88.593 + self.uri = uri
88.594 + self.method = method
88.595 +
88.596 + # uri may be an abs_path (including "http://host.domain.tld");
88.597 + scheme, authority, path = self.parse_request_uri(uri)
88.598 + if '#' in path:
88.599 + self.simple_response("400 Bad Request",
88.600 + "Illegal #fragment in Request-URI.")
88.601 + return
88.602 +
88.603 + if scheme:
88.604 + self.scheme = scheme
88.605 +
88.606 + qs = ''
88.607 + if '?' in path:
88.608 + path, qs = path.split('?', 1)
88.609 +
88.610 + # Unquote the path+params (e.g. "/this%20path" -> "/this path").
88.611 + # http://www.w3.org/Protocols/rfc2616/rfc2616-sec5.html#sec5.1.2
88.612 + #
88.613 + # But note that "...a URI must be separated into its components
88.614 + # before the escaped characters within those components can be
88.615 + # safely decoded." http://www.ietf.org/rfc/rfc2396.txt, sec 2.4.2
88.616 + # Therefore, "/this%2Fpath" becomes "/this%2Fpath", not "/this/path".
88.617 + try:
88.618 + atoms = [unquote(x) for x in quoted_slash.split(path)]
88.619 + except ValueError, ex:
88.620 + self.simple_response("400 Bad Request", ex.args[0])
88.621 + return
88.622 + path = "%2F".join(atoms)
88.623 + self.path = path
88.624 +
88.625 + # Note that, like wsgiref and most other HTTP servers,
88.626 + # we "% HEX HEX"-unquote the path but not the query string.
88.627 + self.qs = qs
88.628 +
88.629 + # Compare request and server HTTP protocol versions, in case our
88.630 + # server does not support the requested protocol. Limit our output
88.631 + # to min(req, server). We want the following output:
88.632 + # request server actual written supported response
88.633 + # protocol protocol response protocol feature set
88.634 + # a 1.0 1.0 1.0 1.0
88.635 + # b 1.0 1.1 1.1 1.0
88.636 + # c 1.1 1.0 1.0 1.0
88.637 + # d 1.1 1.1 1.1 1.1
88.638 + # Notice that, in (b), the response will be "HTTP/1.1" even though
88.639 + # the client only understands 1.0. RFC 2616 10.5.6 says we should
88.640 + # only return 505 if the _major_ version is different.
88.641 + sp = int(self.server.protocol[5]), int(self.server.protocol[7])
88.642 +
88.643 + if sp[0] != rp[0]:
88.644 + self.simple_response("505 HTTP Version Not Supported")
88.645 + return
88.646 + self.request_protocol = req_protocol
88.647 + self.response_protocol = "HTTP/%s.%s" % min(rp, sp)
88.648 +
88.649 + def read_request_headers(self):
88.650 + """Read self.rfile into self.inheaders. Return success."""
88.651 +
88.652 + # then all the http headers
88.653 + try:
88.654 + read_headers(self.rfile, self.inheaders)
88.655 + except ValueError, ex:
88.656 + self.simple_response("400 Bad Request", ex.args[0])
88.657 + return False
88.658 +
88.659 + mrbs = self.server.max_request_body_size
88.660 + if mrbs and int(self.inheaders.get("Content-Length", 0)) > mrbs:
88.661 + self.simple_response("413 Request Entity Too Large",
88.662 + "The entity sent with the request exceeds the maximum "
88.663 + "allowed bytes.")
88.664 + return False
88.665 +
88.666 + # Persistent connection support
88.667 + if self.response_protocol == "HTTP/1.1":
88.668 + # Both server and client are HTTP/1.1
88.669 + if self.inheaders.get("Connection", "") == "close":
88.670 + self.close_connection = True
88.671 + else:
88.672 + # Either the server or client (or both) are HTTP/1.0
88.673 + if self.inheaders.get("Connection", "") != "Keep-Alive":
88.674 + self.close_connection = True
88.675 +
88.676 + # Transfer-Encoding support
88.677 + te = None
88.678 + if self.response_protocol == "HTTP/1.1":
88.679 + te = self.inheaders.get("Transfer-Encoding")
88.680 + if te:
88.681 + te = [x.strip().lower() for x in te.split(",") if x.strip()]
88.682 +
88.683 + self.chunked_read = False
88.684 +
88.685 + if te:
88.686 + for enc in te:
88.687 + if enc == "chunked":
88.688 + self.chunked_read = True
88.689 + else:
88.690 + # Note that, even if we see "chunked", we must reject
88.691 + # if there is an extension we don't recognize.
88.692 + self.simple_response("501 Unimplemented")
88.693 + self.close_connection = True
88.694 + return False
88.695 +
88.696 + # From PEP 333:
88.697 + # "Servers and gateways that implement HTTP 1.1 must provide
88.698 + # transparent support for HTTP 1.1's "expect/continue" mechanism.
88.699 + # This may be done in any of several ways:
88.700 + # 1. Respond to requests containing an Expect: 100-continue request
88.701 + # with an immediate "100 Continue" response, and proceed normally.
88.702 + # 2. Proceed with the request normally, but provide the application
88.703 + # with a wsgi.input stream that will send the "100 Continue"
88.704 + # response if/when the application first attempts to read from
88.705 + # the input stream. The read request must then remain blocked
88.706 + # until the client responds.
88.707 + # 3. Wait until the client decides that the server does not support
88.708 + # expect/continue, and sends the request body on its own.
88.709 + # (This is suboptimal, and is not recommended.)
88.710 + #
88.711 + # We used to do 3, but are now doing 1. Maybe we'll do 2 someday,
88.712 + # but it seems like it would be a big slowdown for such a rare case.
88.713 + if self.inheaders.get("Expect", "") == "100-continue":
88.714 + # Don't use simple_response here, because it emits headers
88.715 + # we don't want. See http://www.cherrypy.org/ticket/951
88.716 + msg = self.server.protocol + " 100 Continue\r\n\r\n"
88.717 + try:
88.718 + self.conn.wfile.sendall(msg)
88.719 + except socket.error, x:
88.720 + if x.args[0] not in socket_errors_to_ignore:
88.721 + raise
88.722 + return True
88.723 +
88.724 + def parse_request_uri(self, uri):
88.725 + """Parse a Request-URI into (scheme, authority, path).
88.726 +
88.727 + Note that Request-URI's must be one of::
88.728 +
88.729 + Request-URI = "*" | absoluteURI | abs_path | authority
88.730 +
88.731 + Therefore, a Request-URI which starts with a double forward-slash
88.732 + cannot be a "net_path"::
88.733 +
88.734 + net_path = "//" authority [ abs_path ]
88.735 +
88.736 + Instead, it must be interpreted as an "abs_path" with an empty first
88.737 + path segment::
88.738 +
88.739 + abs_path = "/" path_segments
88.740 + path_segments = segment *( "/" segment )
88.741 + segment = *pchar *( ";" param )
88.742 + param = *pchar
88.743 + """
88.744 + if uri == "*":
88.745 + return None, None, uri
88.746 +
88.747 + i = uri.find('://')
88.748 + if i > 0 and '?' not in uri[:i]:
88.749 + # An absoluteURI.
88.750 + # If there's a scheme (and it must be http or https), then:
88.751 + # http_URL = "http:" "//" host [ ":" port ] [ abs_path [ "?" query ]]
88.752 + scheme, remainder = uri[:i].lower(), uri[i + 3:]
88.753 + authority, path = remainder.split("/", 1)
88.754 + return scheme, authority, path
88.755 +
88.756 + if uri.startswith('/'):
88.757 + # An abs_path.
88.758 + return None, None, uri
88.759 + else:
88.760 + # An authority.
88.761 + return None, uri, None
88.762 +
88.763 + def respond(self):
88.764 + """Call the gateway and write its iterable output."""
88.765 + mrbs = self.server.max_request_body_size
88.766 + if self.chunked_read:
88.767 + self.rfile = ChunkedRFile(self.conn.rfile, mrbs)
88.768 + else:
88.769 + cl = int(self.inheaders.get("Content-Length", 0))
88.770 + if mrbs and mrbs < cl:
88.771 + if not self.sent_headers:
88.772 + self.simple_response("413 Request Entity Too Large",
88.773 + "The entity sent with the request exceeds the maximum "
88.774 + "allowed bytes.")
88.775 + return
88.776 + self.rfile = KnownLengthRFile(self.conn.rfile, cl)
88.777 +
88.778 + self.server.gateway(self).respond()
88.779 +
88.780 + if (self.ready and not self.sent_headers):
88.781 + self.sent_headers = True
88.782 + self.send_headers()
88.783 + if self.chunked_write:
88.784 + self.conn.wfile.sendall("0\r\n\r\n")
88.785 +
88.786 + def simple_response(self, status, msg=""):
88.787 + """Write a simple response back to the client."""
88.788 + status = str(status)
88.789 + buf = [self.server.protocol + " " +
88.790 + status + CRLF,
88.791 + "Content-Length: %s\r\n" % len(msg),
88.792 + "Content-Type: text/plain\r\n"]
88.793 +
88.794 + if status[:3] in ("413", "414"):
88.795 + # Request Entity Too Large / Request-URI Too Long
88.796 + self.close_connection = True
88.797 + if self.response_protocol == 'HTTP/1.1':
88.798 + # This will not be true for 414, since read_request_line
88.799 + # usually raises 414 before reading the whole line, and we
88.800 + # therefore cannot know the proper response_protocol.
88.801 + buf.append("Connection: close\r\n")
88.802 + else:
88.803 + # HTTP/1.0 had no 413/414 status nor Connection header.
88.804 + # Emit 400 instead and trust the message body is enough.
88.805 + status = "400 Bad Request"
88.806 +
88.807 + buf.append(CRLF)
88.808 + if msg:
88.809 + if isinstance(msg, unicode):
88.810 + msg = msg.encode("ISO-8859-1")
88.811 + buf.append(msg)
88.812 +
88.813 + try:
88.814 + self.conn.wfile.sendall("".join(buf))
88.815 + except socket.error, x:
88.816 + if x.args[0] not in socket_errors_to_ignore:
88.817 + raise
88.818 +
88.819 + def write(self, chunk):
88.820 + """Write unbuffered data to the client."""
88.821 + if self.chunked_write and chunk:
88.822 + buf = [hex(len(chunk))[2:], CRLF, chunk, CRLF]
88.823 + self.conn.wfile.sendall("".join(buf))
88.824 + else:
88.825 + self.conn.wfile.sendall(chunk)
88.826 +
88.827 + def send_headers(self):
88.828 + """Assert, process, and send the HTTP response message-headers.
88.829 +
88.830 + You must set self.status, and self.outheaders before calling this.
88.831 + """
88.832 + hkeys = [key.lower() for key, value in self.outheaders]
88.833 + status = int(self.status[:3])
88.834 +
88.835 + if status == 413:
88.836 + # Request Entity Too Large. Close conn to avoid garbage.
88.837 + self.close_connection = True
88.838 + elif "content-length" not in hkeys:
88.839 + # "All 1xx (informational), 204 (no content),
88.840 + # and 304 (not modified) responses MUST NOT
88.841 + # include a message-body." So no point chunking.
88.842 + if status < 200 or status in (204, 205, 304):
88.843 + pass
88.844 + else:
88.845 + if (self.response_protocol == 'HTTP/1.1'
88.846 + and self.method != 'HEAD'):
88.847 + # Use the chunked transfer-coding
88.848 + self.chunked_write = True
88.849 + self.outheaders.append(("Transfer-Encoding", "chunked"))
88.850 + else:
88.851 + # Closing the conn is the only way to determine len.
88.852 + self.close_connection = True
88.853 +
88.854 + if "connection" not in hkeys:
88.855 + if self.response_protocol == 'HTTP/1.1':
88.856 + # Both server and client are HTTP/1.1 or better
88.857 + if self.close_connection:
88.858 + self.outheaders.append(("Connection", "close"))
88.859 + else:
88.860 + # Server and/or client are HTTP/1.0
88.861 + if not self.close_connection:
88.862 + self.outheaders.append(("Connection", "Keep-Alive"))
88.863 +
88.864 + if (not self.close_connection) and (not self.chunked_read):
88.865 + # Read any remaining request body data on the socket.
88.866 + # "If an origin server receives a request that does not include an
88.867 + # Expect request-header field with the "100-continue" expectation,
88.868 + # the request includes a request body, and the server responds
88.869 + # with a final status code before reading the entire request body
88.870 + # from the transport connection, then the server SHOULD NOT close
88.871 + # the transport connection until it has read the entire request,
88.872 + # or until the client closes the connection. Otherwise, the client
88.873 + # might not reliably receive the response message. However, this
88.874 + # requirement is not be construed as preventing a server from
88.875 + # defending itself against denial-of-service attacks, or from
88.876 + # badly broken client implementations."
88.877 + remaining = getattr(self.rfile, 'remaining', 0)
88.878 + if remaining > 0:
88.879 + self.rfile.read(remaining)
88.880 +
88.881 + if "date" not in hkeys:
88.882 + self.outheaders.append(("Date", rfc822.formatdate()))
88.883 +
88.884 + if "server" not in hkeys:
88.885 + self.outheaders.append(("Server", self.server.server_name))
88.886 +
88.887 + buf = [self.server.protocol + " " + self.status + CRLF]
88.888 + for k, v in self.outheaders:
88.889 + buf.append(k + ": " + v + CRLF)
88.890 + buf.append(CRLF)
88.891 + self.conn.wfile.sendall("".join(buf))
88.892 +
88.893 +
88.894 +class NoSSLError(Exception):
88.895 + """Exception raised when a client speaks HTTP to an HTTPS socket."""
88.896 + pass
88.897 +
88.898 +
88.899 +class FatalSSLAlert(Exception):
88.900 + """Exception raised when the SSL implementation signals a fatal alert."""
88.901 + pass
88.902 +
88.903 +
88.904 +class CP_fileobject(socket._fileobject):
88.905 + """Faux file object attached to a socket object."""
88.906 +
88.907 + def __init__(self, *args, **kwargs):
88.908 + self.bytes_read = 0
88.909 + self.bytes_written = 0
88.910 + socket._fileobject.__init__(self, *args, **kwargs)
88.911 +
88.912 + def sendall(self, data):
88.913 + """Sendall for non-blocking sockets."""
88.914 + while data:
88.915 + try:
88.916 + bytes_sent = self.send(data)
88.917 + data = data[bytes_sent:]
88.918 + except socket.error, e:
88.919 + if e.args[0] not in socket_errors_nonblocking:
88.920 + raise
88.921 +
88.922 + def send(self, data):
88.923 + bytes_sent = self._sock.send(data)
88.924 + self.bytes_written += bytes_sent
88.925 + return bytes_sent
88.926 +
88.927 + def flush(self):
88.928 + if self._wbuf:
88.929 + buffer = "".join(self._wbuf)
88.930 + self._wbuf = []
88.931 + self.sendall(buffer)
88.932 +
88.933 + def recv(self, size):
88.934 + while True:
88.935 + try:
88.936 + data = self._sock.recv(size)
88.937 + self.bytes_read += len(data)
88.938 + return data
88.939 + except socket.error, e:
88.940 + if (e.args[0] not in socket_errors_nonblocking
88.941 + and e.args[0] not in socket_error_eintr):
88.942 + raise
88.943 +
88.944 + if not _fileobject_uses_str_type:
88.945 + def read(self, size=-1):
88.946 + # Use max, disallow tiny reads in a loop as they are very inefficient.
88.947 + # We never leave read() with any leftover data from a new recv() call
88.948 + # in our internal buffer.
88.949 + rbufsize = max(self._rbufsize, self.default_bufsize)
88.950 + # Our use of StringIO rather than lists of string objects returned by
88.951 + # recv() minimizes memory usage and fragmentation that occurs when
88.952 + # rbufsize is large compared to the typical return value of recv().
88.953 + buf = self._rbuf
88.954 + buf.seek(0, 2) # seek end
88.955 + if size < 0:
88.956 + # Read until EOF
88.957 + self._rbuf = StringIO.StringIO() # reset _rbuf. we consume it via buf.
88.958 + while True:
88.959 + data = self.recv(rbufsize)
88.960 + if not data:
88.961 + break
88.962 + buf.write(data)
88.963 + return buf.getvalue()
88.964 + else:
88.965 + # Read until size bytes or EOF seen, whichever comes first
88.966 + buf_len = buf.tell()
88.967 + if buf_len >= size:
88.968 + # Already have size bytes in our buffer? Extract and return.
88.969 + buf.seek(0)
88.970 + rv = buf.read(size)
88.971 + self._rbuf = StringIO.StringIO()
88.972 + self._rbuf.write(buf.read())
88.973 + return rv
88.974 +
88.975 + self._rbuf = StringIO.StringIO() # reset _rbuf. we consume it via buf.
88.976 + while True:
88.977 + left = size - buf_len
88.978 + # recv() will malloc the amount of memory given as its
88.979 + # parameter even though it often returns much less data
88.980 + # than that. The returned data string is short lived
88.981 + # as we copy it into a StringIO and free it. This avoids
88.982 + # fragmentation issues on many platforms.
88.983 + data = self.recv(left)
88.984 + if not data:
88.985 + break
88.986 + n = len(data)
88.987 + if n == size and not buf_len:
88.988 + # Shortcut. Avoid buffer data copies when:
88.989 + # - We have no data in our buffer.
88.990 + # AND
88.991 + # - Our call to recv returned exactly the
88.992 + # number of bytes we were asked to read.
88.993 + return data
88.994 + if n == left:
88.995 + buf.write(data)
88.996 + del data # explicit free
88.997 + break
88.998 + assert n <= left, "recv(%d) returned %d bytes" % (left, n)
88.999 + buf.write(data)
88.1000 + buf_len += n
88.1001 + del data # explicit free
88.1002 + #assert buf_len == buf.tell()
88.1003 + return buf.getvalue()
88.1004 +
88.1005 + def readline(self, size=-1):
88.1006 + buf = self._rbuf
88.1007 + buf.seek(0, 2) # seek end
88.1008 + if buf.tell() > 0:
88.1009 + # check if we already have it in our buffer
88.1010 + buf.seek(0)
88.1011 + bline = buf.readline(size)
88.1012 + if bline.endswith('\n') or len(bline) == size:
88.1013 + self._rbuf = StringIO.StringIO()
88.1014 + self._rbuf.write(buf.read())
88.1015 + return bline
88.1016 + del bline
88.1017 + if size < 0:
88.1018 + # Read until \n or EOF, whichever comes first
88.1019 + if self._rbufsize <= 1:
88.1020 + # Speed up unbuffered case
88.1021 + buf.seek(0)
88.1022 + buffers = [buf.read()]
88.1023 + self._rbuf = StringIO.StringIO() # reset _rbuf. we consume it via buf.
88.1024 + data = None
88.1025 + recv = self.recv
88.1026 + while data != "\n":
88.1027 + data = recv(1)
88.1028 + if not data:
88.1029 + break
88.1030 + buffers.append(data)
88.1031 + return "".join(buffers)
88.1032 +
88.1033 + buf.seek(0, 2) # seek end
88.1034 + self._rbuf = StringIO.StringIO() # reset _rbuf. we consume it via buf.
88.1035 + while True:
88.1036 + data = self.recv(self._rbufsize)
88.1037 + if not data:
88.1038 + break
88.1039 + nl = data.find('\n')
88.1040 + if nl >= 0:
88.1041 + nl += 1
88.1042 + buf.write(data[:nl])
88.1043 + self._rbuf.write(data[nl:])
88.1044 + del data
88.1045 + break
88.1046 + buf.write(data)
88.1047 + return buf.getvalue()
88.1048 + else:
88.1049 + # Read until size bytes or \n or EOF seen, whichever comes first
88.1050 + buf.seek(0, 2) # seek end
88.1051 + buf_len = buf.tell()
88.1052 + if buf_len >= size:
88.1053 + buf.seek(0)
88.1054 + rv = buf.read(size)
88.1055 + self._rbuf = StringIO.StringIO()
88.1056 + self._rbuf.write(buf.read())
88.1057 + return rv
88.1058 + self._rbuf = StringIO.StringIO() # reset _rbuf. we consume it via buf.
88.1059 + while True:
88.1060 + data = self.recv(self._rbufsize)
88.1061 + if not data:
88.1062 + break
88.1063 + left = size - buf_len
88.1064 + # did we just receive a newline?
88.1065 + nl = data.find('\n', 0, left)
88.1066 + if nl >= 0:
88.1067 + nl += 1
88.1068 + # save the excess data to _rbuf
88.1069 + self._rbuf.write(data[nl:])
88.1070 + if buf_len:
88.1071 + buf.write(data[:nl])
88.1072 + break
88.1073 + else:
88.1074 + # Shortcut. Avoid data copy through buf when returning
88.1075 + # a substring of our first recv().
88.1076 + return data[:nl]
88.1077 + n = len(data)
88.1078 + if n == size and not buf_len:
88.1079 + # Shortcut. Avoid data copy through buf when
88.1080 + # returning exactly all of our first recv().
88.1081 + return data
88.1082 + if n >= left:
88.1083 + buf.write(data[:left])
88.1084 + self._rbuf.write(data[left:])
88.1085 + break
88.1086 + buf.write(data)
88.1087 + buf_len += n
88.1088 + #assert buf_len == buf.tell()
88.1089 + return buf.getvalue()
88.1090 + else:
88.1091 + def read(self, size=-1):
88.1092 + if size < 0:
88.1093 + # Read until EOF
88.1094 + buffers = [self._rbuf]
88.1095 + self._rbuf = ""
88.1096 + if self._rbufsize <= 1:
88.1097 + recv_size = self.default_bufsize
88.1098 + else:
88.1099 + recv_size = self._rbufsize
88.1100 +
88.1101 + while True:
88.1102 + data = self.recv(recv_size)
88.1103 + if not data:
88.1104 + break
88.1105 + buffers.append(data)
88.1106 + return "".join(buffers)
88.1107 + else:
88.1108 + # Read until size bytes or EOF seen, whichever comes first
88.1109 + data = self._rbuf
88.1110 + buf_len = len(data)
88.1111 + if buf_len >= size:
88.1112 + self._rbuf = data[size:]
88.1113 + return data[:size]
88.1114 + buffers = []
88.1115 + if data:
88.1116 + buffers.append(data)
88.1117 + self._rbuf = ""
88.1118 + while True:
88.1119 + left = size - buf_len
88.1120 + recv_size = max(self._rbufsize, left)
88.1121 + data = self.recv(recv_size)
88.1122 + if not data:
88.1123 + break
88.1124 + buffers.append(data)
88.1125 + n = len(data)
88.1126 + if n >= left:
88.1127 + self._rbuf = data[left:]
88.1128 + buffers[-1] = data[:left]
88.1129 + break
88.1130 + buf_len += n
88.1131 + return "".join(buffers)
88.1132 +
88.1133 + def readline(self, size=-1):
88.1134 + data = self._rbuf
88.1135 + if size < 0:
88.1136 + # Read until \n or EOF, whichever comes first
88.1137 + if self._rbufsize <= 1:
88.1138 + # Speed up unbuffered case
88.1139 + assert data == ""
88.1140 + buffers = []
88.1141 + while data != "\n":
88.1142 + data = self.recv(1)
88.1143 + if not data:
88.1144 + break
88.1145 + buffers.append(data)
88.1146 + return "".join(buffers)
88.1147 + nl = data.find('\n')
88.1148 + if nl >= 0:
88.1149 + nl += 1
88.1150 + self._rbuf = data[nl:]
88.1151 + return data[:nl]
88.1152 + buffers = []
88.1153 + if data:
88.1154 + buffers.append(data)
88.1155 + self._rbuf = ""
88.1156 + while True:
88.1157 + data = self.recv(self._rbufsize)
88.1158 + if not data:
88.1159 + break
88.1160 + buffers.append(data)
88.1161 + nl = data.find('\n')
88.1162 + if nl >= 0:
88.1163 + nl += 1
88.1164 + self._rbuf = data[nl:]
88.1165 + buffers[-1] = data[:nl]
88.1166 + break
88.1167 + return "".join(buffers)
88.1168 + else:
88.1169 + # Read until size bytes or \n or EOF seen, whichever comes first
88.1170 + nl = data.find('\n', 0, size)
88.1171 + if nl >= 0:
88.1172 + nl += 1
88.1173 + self._rbuf = data[nl:]
88.1174 + return data[:nl]
88.1175 + buf_len = len(data)
88.1176 + if buf_len >= size:
88.1177 + self._rbuf = data[size:]
88.1178 + return data[:size]
88.1179 + buffers = []
88.1180 + if data:
88.1181 + buffers.append(data)
88.1182 + self._rbuf = ""
88.1183 + while True:
88.1184 + data = self.recv(self._rbufsize)
88.1185 + if not data:
88.1186 + break
88.1187 + buffers.append(data)
88.1188 + left = size - buf_len
88.1189 + nl = data.find('\n', 0, left)
88.1190 + if nl >= 0:
88.1191 + nl += 1
88.1192 + self._rbuf = data[nl:]
88.1193 + buffers[-1] = data[:nl]
88.1194 + break
88.1195 + n = len(data)
88.1196 + if n >= left:
88.1197 + self._rbuf = data[left:]
88.1198 + buffers[-1] = data[:left]
88.1199 + break
88.1200 + buf_len += n
88.1201 + return "".join(buffers)
88.1202 +
88.1203 +
88.1204 +class HTTPConnection(object):
88.1205 + """An HTTP connection (active socket).
88.1206 +
88.1207 + server: the Server object which received this connection.
88.1208 + socket: the raw socket object (usually TCP) for this connection.
88.1209 + makefile: a fileobject class for reading from the socket.
88.1210 + """
88.1211 +
88.1212 + remote_addr = None
88.1213 + remote_port = None
88.1214 + ssl_env = None
88.1215 + rbufsize = DEFAULT_BUFFER_SIZE
88.1216 + wbufsize = DEFAULT_BUFFER_SIZE
88.1217 + RequestHandlerClass = HTTPRequest
88.1218 +
88.1219 + def __init__(self, server, sock, makefile=CP_fileobject):
88.1220 + self.server = server
88.1221 + self.socket = sock
88.1222 + self.rfile = makefile(sock, "rb", self.rbufsize)
88.1223 + self.wfile = makefile(sock, "wb", self.wbufsize)
88.1224 + self.requests_seen = 0
88.1225 +
88.1226 + def communicate(self):
88.1227 + """Read each request and respond appropriately."""
88.1228 + request_seen = False
88.1229 + try:
88.1230 + while True:
88.1231 + # (re)set req to None so that if something goes wrong in
88.1232 + # the RequestHandlerClass constructor, the error doesn't
88.1233 + # get written to the previous request.
88.1234 + req = None
88.1235 + req = self.RequestHandlerClass(self.server, self)
88.1236 +
88.1237 + # This order of operations should guarantee correct pipelining.
88.1238 + req.parse_request()
88.1239 + if self.server.stats['Enabled']:
88.1240 + self.requests_seen += 1
88.1241 + if not req.ready:
88.1242 + # Something went wrong in the parsing (and the server has
88.1243 + # probably already made a simple_response). Return and
88.1244 + # let the conn close.
88.1245 + return
88.1246 +
88.1247 + request_seen = True
88.1248 + req.respond()
88.1249 + if req.close_connection:
88.1250 + return
88.1251 + except socket.error, e:
88.1252 + errnum = e.args[0]
88.1253 + # sadly SSL sockets return a different (longer) time out string
88.1254 + if errnum == 'timed out' or errnum == 'The read operation timed out':
88.1255 + # Don't error if we're between requests; only error
88.1256 + # if 1) no request has been started at all, or 2) we're
88.1257 + # in the middle of a request.
88.1258 + # See http://www.cherrypy.org/ticket/853
88.1259 + if (not request_seen) or (req and req.started_request):
88.1260 + # Don't bother writing the 408 if the response
88.1261 + # has already started being written.
88.1262 + if req and not req.sent_headers:
88.1263 + try:
88.1264 + req.simple_response("408 Request Timeout")
88.1265 + except FatalSSLAlert:
88.1266 + # Close the connection.
88.1267 + return
88.1268 + elif errnum not in socket_errors_to_ignore:
88.1269 + if req and not req.sent_headers:
88.1270 + try:
88.1271 + req.simple_response("500 Internal Server Error",
88.1272 + format_exc())
88.1273 + except FatalSSLAlert:
88.1274 + # Close the connection.
88.1275 + return
88.1276 + return
88.1277 + except (KeyboardInterrupt, SystemExit):
88.1278 + raise
88.1279 + except FatalSSLAlert:
88.1280 + # Close the connection.
88.1281 + return
88.1282 + except NoSSLError:
88.1283 + if req and not req.sent_headers:
88.1284 + # Unwrap our wfile
88.1285 + self.wfile = CP_fileobject(self.socket._sock, "wb", self.wbufsize)
88.1286 + req.simple_response("400 Bad Request",
88.1287 + "The client sent a plain HTTP request, but "
88.1288 + "this server only speaks HTTPS on this port.")
88.1289 + self.linger = True
88.1290 + except Exception:
88.1291 + if req and not req.sent_headers:
88.1292 + try:
88.1293 + req.simple_response("500 Internal Server Error", format_exc())
88.1294 + except FatalSSLAlert:
88.1295 + # Close the connection.
88.1296 + return
88.1297 +
88.1298 + linger = False
88.1299 +
88.1300 + def close(self):
88.1301 + """Close the socket underlying this connection."""
88.1302 + self.rfile.close()
88.1303 +
88.1304 + if not self.linger:
88.1305 + # Python's socket module does NOT call close on the kernel socket
88.1306 + # when you call socket.close(). We do so manually here because we
88.1307 + # want this server to send a FIN TCP segment immediately. Note this
88.1308 + # must be called *before* calling socket.close(), because the latter
88.1309 + # drops its reference to the kernel socket.
88.1310 + if hasattr(self.socket, '_sock'):
88.1311 + self.socket._sock.close()
88.1312 + self.socket.close()
88.1313 + else:
88.1314 + # On the other hand, sometimes we want to hang around for a bit
88.1315 + # to make sure the client has a chance to read our entire
88.1316 + # response. Skipping the close() calls here delays the FIN
88.1317 + # packet until the socket object is garbage-collected later.
88.1318 + # Someday, perhaps, we'll do the full lingering_close that
88.1319 + # Apache does, but not today.
88.1320 + pass
88.1321 +
88.1322 +
88.1323 +_SHUTDOWNREQUEST = None
88.1324 +
88.1325 +class WorkerThread(threading.Thread):
88.1326 + """Thread which continuously polls a Queue for Connection objects.
88.1327 +
88.1328 + Due to the timing issues of polling a Queue, a WorkerThread does not
88.1329 + check its own 'ready' flag after it has started. To stop the thread,
88.1330 + it is necessary to stick a _SHUTDOWNREQUEST object onto the Queue
88.1331 + (one for each running WorkerThread).
88.1332 + """
88.1333 +
88.1334 + conn = None
88.1335 + """The current connection pulled off the Queue, or None."""
88.1336 +
88.1337 + server = None
88.1338 + """The HTTP Server which spawned this thread, and which owns the
88.1339 + Queue and is placing active connections into it."""
88.1340 +
88.1341 + ready = False
88.1342 + """A simple flag for the calling server to know when this thread
88.1343 + has begun polling the Queue."""
88.1344 +
88.1345 +
88.1346 + def __init__(self, server):
88.1347 + self.ready = False
88.1348 + self.server = server
88.1349 +
88.1350 + self.requests_seen = 0
88.1351 + self.bytes_read = 0
88.1352 + self.bytes_written = 0
88.1353 + self.start_time = None
88.1354 + self.work_time = 0
88.1355 + self.stats = {
88.1356 + 'Requests': lambda s: self.requests_seen + ((self.start_time is None) and 0 or self.conn.requests_seen),
88.1357 + 'Bytes Read': lambda s: self.bytes_read + ((self.start_time is None) and 0 or self.conn.rfile.bytes_read),
88.1358 + 'Bytes Written': lambda s: self.bytes_written + ((self.start_time is None) and 0 or self.conn.wfile.bytes_written),
88.1359 + 'Work Time': lambda s: self.work_time + ((self.start_time is None) and 0 or time.time() - self.start_time),
88.1360 + 'Read Throughput': lambda s: s['Bytes Read'](s) / (s['Work Time'](s) or 1e-6),
88.1361 + 'Write Throughput': lambda s: s['Bytes Written'](s) / (s['Work Time'](s) or 1e-6),
88.1362 + }
88.1363 + threading.Thread.__init__(self)
88.1364 +
88.1365 + def run(self):
88.1366 + self.server.stats['Worker Threads'][self.getName()] = self.stats
88.1367 + try:
88.1368 + self.ready = True
88.1369 + while True:
88.1370 + conn = self.server.requests.get()
88.1371 + if conn is _SHUTDOWNREQUEST:
88.1372 + return
88.1373 +
88.1374 + self.conn = conn
88.1375 + if self.server.stats['Enabled']:
88.1376 + self.start_time = time.time()
88.1377 + try:
88.1378 + conn.communicate()
88.1379 + finally:
88.1380 + conn.close()
88.1381 + if self.server.stats['Enabled']:
88.1382 + self.requests_seen += self.conn.requests_seen
88.1383 + self.bytes_read += self.conn.rfile.bytes_read
88.1384 + self.bytes_written += self.conn.wfile.bytes_written
88.1385 + self.work_time += time.time() - self.start_time
88.1386 + self.start_time = None
88.1387 + self.conn = None
88.1388 + except (KeyboardInterrupt, SystemExit), exc:
88.1389 + self.server.interrupt = exc
88.1390 +
88.1391 +
88.1392 +class ThreadPool(object):
88.1393 + """A Request Queue for the CherryPyWSGIServer which pools threads.
88.1394 +
88.1395 + ThreadPool objects must provide min, get(), put(obj), start()
88.1396 + and stop(timeout) attributes.
88.1397 + """
88.1398 +
88.1399 + def __init__(self, server, min=10, max=-1):
88.1400 + self.server = server
88.1401 + self.min = min
88.1402 + self.max = max
88.1403 + self._threads = []
88.1404 + self._queue = Queue.Queue()
88.1405 + self.get = self._queue.get
88.1406 +
88.1407 + def start(self):
88.1408 + """Start the pool of threads."""
88.1409 + for i in range(self.min):
88.1410 + self._threads.append(WorkerThread(self.server))
88.1411 + for worker in self._threads:
88.1412 + worker.setName("CP Server " + worker.getName())
88.1413 + worker.start()
88.1414 + for worker in self._threads:
88.1415 + while not worker.ready:
88.1416 + time.sleep(.1)
88.1417 +
88.1418 + def _get_idle(self):
88.1419 + """Number of worker threads which are idle. Read-only."""
88.1420 + return len([t for t in self._threads if t.conn is None])
88.1421 + idle = property(_get_idle, doc=_get_idle.__doc__)
88.1422 +
88.1423 + def put(self, obj):
88.1424 + self._queue.put(obj)
88.1425 + if obj is _SHUTDOWNREQUEST:
88.1426 + return
88.1427 +
88.1428 + def grow(self, amount):
88.1429 + """Spawn new worker threads (not above self.max)."""
88.1430 + for i in range(amount):
88.1431 + if self.max > 0 and len(self._threads) >= self.max:
88.1432 + break
88.1433 + worker = WorkerThread(self.server)
88.1434 + worker.setName("CP Server " + worker.getName())
88.1435 + self._threads.append(worker)
88.1436 + worker.start()
88.1437 +
88.1438 + def shrink(self, amount):
88.1439 + """Kill off worker threads (not below self.min)."""
88.1440 + # Grow/shrink the pool if necessary.
88.1441 + # Remove any dead threads from our list
88.1442 + for t in self._threads:
88.1443 + if not t.isAlive():
88.1444 + self._threads.remove(t)
88.1445 + amount -= 1
88.1446 +
88.1447 + if amount > 0:
88.1448 + for i in range(min(amount, len(self._threads) - self.min)):
88.1449 + # Put a number of shutdown requests on the queue equal
88.1450 + # to 'amount'. Once each of those is processed by a worker,
88.1451 + # that worker will terminate and be culled from our list
88.1452 + # in self.put.
88.1453 + self._queue.put(_SHUTDOWNREQUEST)
88.1454 +
88.1455 + def stop(self, timeout=5):
88.1456 + # Must shut down threads here so the code that calls
88.1457 + # this method can know when all threads are stopped.
88.1458 + for worker in self._threads:
88.1459 + self._queue.put(_SHUTDOWNREQUEST)
88.1460 +
88.1461 + # Don't join currentThread (when stop is called inside a request).
88.1462 + current = threading.currentThread()
88.1463 + if timeout and timeout >= 0:
88.1464 + endtime = time.time() + timeout
88.1465 + while self._threads:
88.1466 + worker = self._threads.pop()
88.1467 + if worker is not current and worker.isAlive():
88.1468 + try:
88.1469 + if timeout is None or timeout < 0:
88.1470 + worker.join()
88.1471 + else:
88.1472 + remaining_time = endtime - time.time()
88.1473 + if remaining_time > 0:
88.1474 + worker.join(remaining_time)
88.1475 + if worker.isAlive():
88.1476 + # We exhausted the timeout.
88.1477 + # Forcibly shut down the socket.
88.1478 + c = worker.conn
88.1479 + if c and not c.rfile.closed:
88.1480 + try:
88.1481 + c.socket.shutdown(socket.SHUT_RD)
88.1482 + except TypeError:
88.1483 + # pyOpenSSL sockets don't take an arg
88.1484 + c.socket.shutdown()
88.1485 + worker.join()
88.1486 + except (AssertionError,
88.1487 + # Ignore repeated Ctrl-C.
88.1488 + # See http://www.cherrypy.org/ticket/691.
88.1489 + KeyboardInterrupt), exc1:
88.1490 + pass
88.1491 +
88.1492 + def _get_qsize(self):
88.1493 + return self._queue.qsize()
88.1494 + qsize = property(_get_qsize)
88.1495 +
88.1496 +
88.1497 +
88.1498 +try:
88.1499 + import fcntl
88.1500 +except ImportError:
88.1501 + try:
88.1502 + from ctypes import windll, WinError
88.1503 + except ImportError:
88.1504 + def prevent_socket_inheritance(sock):
88.1505 + """Dummy function, since neither fcntl nor ctypes are available."""
88.1506 + pass
88.1507 + else:
88.1508 + def prevent_socket_inheritance(sock):
88.1509 + """Mark the given socket fd as non-inheritable (Windows)."""
88.1510 + if not windll.kernel32.SetHandleInformation(sock.fileno(), 1, 0):
88.1511 + raise WinError()
88.1512 +else:
88.1513 + def prevent_socket_inheritance(sock):
88.1514 + """Mark the given socket fd as non-inheritable (POSIX)."""
88.1515 + fd = sock.fileno()
88.1516 + old_flags = fcntl.fcntl(fd, fcntl.F_GETFD)
88.1517 + fcntl.fcntl(fd, fcntl.F_SETFD, old_flags | fcntl.FD_CLOEXEC)
88.1518 +
88.1519 +
88.1520 +class SSLAdapter(object):
88.1521 + """Base class for SSL driver library adapters.
88.1522 +
88.1523 + Required methods:
88.1524 +
88.1525 + * ``wrap(sock) -> (wrapped socket, ssl environ dict)``
88.1526 + * ``makefile(sock, mode='r', bufsize=DEFAULT_BUFFER_SIZE) -> socket file object``
88.1527 + """
88.1528 +
88.1529 + def __init__(self, certificate, private_key, certificate_chain=None):
88.1530 + self.certificate = certificate
88.1531 + self.private_key = private_key
88.1532 + self.certificate_chain = certificate_chain
88.1533 +
88.1534 + def wrap(self, sock):
88.1535 + raise NotImplemented
88.1536 +
88.1537 + def makefile(self, sock, mode='r', bufsize=DEFAULT_BUFFER_SIZE):
88.1538 + raise NotImplemented
88.1539 +
88.1540 +
88.1541 +class HTTPServer(object):
88.1542 + """An HTTP server."""
88.1543 +
88.1544 + _bind_addr = "127.0.0.1"
88.1545 + _interrupt = None
88.1546 +
88.1547 + gateway = None
88.1548 + """A Gateway instance."""
88.1549 +
88.1550 + minthreads = None
88.1551 + """The minimum number of worker threads to create (default 10)."""
88.1552 +
88.1553 + maxthreads = None
88.1554 + """The maximum number of worker threads to create (default -1 = no limit)."""
88.1555 +
88.1556 + server_name = None
88.1557 + """The name of the server; defaults to socket.gethostname()."""
88.1558 +
88.1559 + protocol = "HTTP/1.1"
88.1560 + """The version string to write in the Status-Line of all HTTP responses.
88.1561 +
88.1562 + For example, "HTTP/1.1" is the default. This also limits the supported
88.1563 + features used in the response."""
88.1564 +
88.1565 + request_queue_size = 5
88.1566 + """The 'backlog' arg to socket.listen(); max queued connections (default 5)."""
88.1567 +
88.1568 + shutdown_timeout = 5
88.1569 + """The total time, in seconds, to wait for worker threads to cleanly exit."""
88.1570 +
88.1571 + timeout = 10
88.1572 + """The timeout in seconds for accepted connections (default 10)."""
88.1573 +
88.1574 + version = "CherryPy/3.2.0"
88.1575 + """A version string for the HTTPServer."""
88.1576 +
88.1577 + software = None
88.1578 + """The value to set for the SERVER_SOFTWARE entry in the WSGI environ.
88.1579 +
88.1580 + If None, this defaults to ``'%s Server' % self.version``."""
88.1581 +
88.1582 + ready = False
88.1583 + """An internal flag which marks whether the socket is accepting connections."""
88.1584 +
88.1585 + max_request_header_size = 0
88.1586 + """The maximum size, in bytes, for request headers, or 0 for no limit."""
88.1587 +
88.1588 + max_request_body_size = 0
88.1589 + """The maximum size, in bytes, for request bodies, or 0 for no limit."""
88.1590 +
88.1591 + nodelay = True
88.1592 + """If True (the default since 3.1), sets the TCP_NODELAY socket option."""
88.1593 +
88.1594 + ConnectionClass = HTTPConnection
88.1595 + """The class to use for handling HTTP connections."""
88.1596 +
88.1597 + ssl_adapter = None
88.1598 + """An instance of SSLAdapter (or a subclass).
88.1599 +
88.1600 + You must have the corresponding SSL driver library installed."""
88.1601 +
88.1602 + def __init__(self, bind_addr, gateway, minthreads=10, maxthreads=-1,
88.1603 + server_name=None):
88.1604 + self.bind_addr = bind_addr
88.1605 + self.gateway = gateway
88.1606 +
88.1607 + self.requests = ThreadPool(self, min=minthreads or 1, max=maxthreads)
88.1608 +
88.1609 + if not server_name:
88.1610 + server_name = socket.gethostname()
88.1611 + self.server_name = server_name
88.1612 + self.clear_stats()
88.1613 +
88.1614 + def clear_stats(self):
88.1615 + self._start_time = None
88.1616 + self._run_time = 0
88.1617 + self.stats = {
88.1618 + 'Enabled': False,
88.1619 + 'Bind Address': lambda s: repr(self.bind_addr),
88.1620 + 'Run time': lambda s: (not s['Enabled']) and 0 or self.runtime(),
88.1621 + 'Accepts': 0,
88.1622 + 'Accepts/sec': lambda s: s['Accepts'] / self.runtime(),
88.1623 + 'Queue': lambda s: getattr(self.requests, "qsize", None),
88.1624 + 'Threads': lambda s: len(getattr(self.requests, "_threads", [])),
88.1625 + 'Threads Idle': lambda s: getattr(self.requests, "idle", None),
88.1626 + 'Socket Errors': 0,
88.1627 + 'Requests': lambda s: (not s['Enabled']) and 0 or sum([w['Requests'](w) for w
88.1628 + in s['Worker Threads'].values()], 0),
88.1629 + 'Bytes Read': lambda s: (not s['Enabled']) and 0 or sum([w['Bytes Read'](w) for w
88.1630 + in s['Worker Threads'].values()], 0),
88.1631 + 'Bytes Written': lambda s: (not s['Enabled']) and 0 or sum([w['Bytes Written'](w) for w
88.1632 + in s['Worker Threads'].values()], 0),
88.1633 + 'Work Time': lambda s: (not s['Enabled']) and 0 or sum([w['Work Time'](w) for w
88.1634 + in s['Worker Threads'].values()], 0),
88.1635 + 'Read Throughput': lambda s: (not s['Enabled']) and 0 or sum(
88.1636 + [w['Bytes Read'](w) / (w['Work Time'](w) or 1e-6)
88.1637 + for w in s['Worker Threads'].values()], 0),
88.1638 + 'Write Throughput': lambda s: (not s['Enabled']) and 0 or sum(
88.1639 + [w['Bytes Written'](w) / (w['Work Time'](w) or 1e-6)
88.1640 + for w in s['Worker Threads'].values()], 0),
88.1641 + 'Worker Threads': {},
88.1642 + }
88.1643 + logging.statistics["CherryPy HTTPServer %d" % id(self)] = self.stats
88.1644 +
88.1645 + def runtime(self):
88.1646 + if self._start_time is None:
88.1647 + return self._run_time
88.1648 + else:
88.1649 + return self._run_time + (time.time() - self._start_time)
88.1650 +
88.1651 + def __str__(self):
88.1652 + return "%s.%s(%r)" % (self.__module__, self.__class__.__name__,
88.1653 + self.bind_addr)
88.1654 +
88.1655 + def _get_bind_addr(self):
88.1656 + return self._bind_addr
88.1657 + def _set_bind_addr(self, value):
88.1658 + if isinstance(value, tuple) and value[0] in ('', None):
88.1659 + # Despite the socket module docs, using '' does not
88.1660 + # allow AI_PASSIVE to work. Passing None instead
88.1661 + # returns '0.0.0.0' like we want. In other words:
88.1662 + # host AI_PASSIVE result
88.1663 + # '' Y 192.168.x.y
88.1664 + # '' N 192.168.x.y
88.1665 + # None Y 0.0.0.0
88.1666 + # None N 127.0.0.1
88.1667 + # But since you can get the same effect with an explicit
88.1668 + # '0.0.0.0', we deny both the empty string and None as values.
88.1669 + raise ValueError("Host values of '' or None are not allowed. "
88.1670 + "Use '0.0.0.0' (IPv4) or '::' (IPv6) instead "
88.1671 + "to listen on all active interfaces.")
88.1672 + self._bind_addr = value
88.1673 + bind_addr = property(_get_bind_addr, _set_bind_addr,
88.1674 + doc="""The interface on which to listen for connections.
88.1675 +
88.1676 + For TCP sockets, a (host, port) tuple. Host values may be any IPv4
88.1677 + or IPv6 address, or any valid hostname. The string 'localhost' is a
88.1678 + synonym for '127.0.0.1' (or '::1', if your hosts file prefers IPv6).
88.1679 + The string '0.0.0.0' is a special IPv4 entry meaning "any active
88.1680 + interface" (INADDR_ANY), and '::' is the similar IN6ADDR_ANY for
88.1681 + IPv6. The empty string or None are not allowed.
88.1682 +
88.1683 + For UNIX sockets, supply the filename as a string.""")
88.1684 +
88.1685 + def start(self):
88.1686 + """Run the server forever."""
88.1687 + # We don't have to trap KeyboardInterrupt or SystemExit here,
88.1688 + # because cherrpy.server already does so, calling self.stop() for us.
88.1689 + # If you're using this server with another framework, you should
88.1690 + # trap those exceptions in whatever code block calls start().
88.1691 + self._interrupt = None
88.1692 +
88.1693 + if self.software is None:
88.1694 + self.software = "%s Server" % self.version
88.1695 +
88.1696 + # SSL backward compatibility
88.1697 + if (self.ssl_adapter is None and
88.1698 + getattr(self, 'ssl_certificate', None) and
88.1699 + getattr(self, 'ssl_private_key', None)):
88.1700 + warnings.warn(
88.1701 + "SSL attributes are deprecated in CherryPy 3.2, and will "
88.1702 + "be removed in CherryPy 3.3. Use an ssl_adapter attribute "
88.1703 + "instead.",
88.1704 + DeprecationWarning
88.1705 + )
88.1706 + try:
88.1707 + from cherrypy.wsgiserver.ssl_pyopenssl import pyOpenSSLAdapter
88.1708 + except ImportError:
88.1709 + pass
88.1710 + else:
88.1711 + self.ssl_adapter = pyOpenSSLAdapter(
88.1712 + self.ssl_certificate, self.ssl_private_key,
88.1713 + getattr(self, 'ssl_certificate_chain', None))
88.1714 +
88.1715 + # Select the appropriate socket
88.1716 + if isinstance(self.bind_addr, basestring):
88.1717 + # AF_UNIX socket
88.1718 +
88.1719 + # So we can reuse the socket...
88.1720 + try: os.unlink(self.bind_addr)
88.1721 + except: pass
88.1722 +
88.1723 + # So everyone can access the socket...
88.1724 + try: os.chmod(self.bind_addr, 0777)
88.1725 + except: pass
88.1726 +
88.1727 + info = [(socket.AF_UNIX, socket.SOCK_STREAM, 0, "", self.bind_addr)]
88.1728 + else:
88.1729 + # AF_INET or AF_INET6 socket
88.1730 + # Get the correct address family for our host (allows IPv6 addresses)
88.1731 + host, port = self.bind_addr
88.1732 + try:
88.1733 + info = socket.getaddrinfo(host, port, socket.AF_UNSPEC,
88.1734 + socket.SOCK_STREAM, 0, socket.AI_PASSIVE)
88.1735 + except socket.gaierror:
88.1736 + if ':' in self.bind_addr[0]:
88.1737 + info = [(socket.AF_INET6, socket.SOCK_STREAM,
88.1738 + 0, "", self.bind_addr + (0, 0))]
88.1739 + else:
88.1740 + info = [(socket.AF_INET, socket.SOCK_STREAM,
88.1741 + 0, "", self.bind_addr)]
88.1742 +
88.1743 + self.socket = None
88.1744 + msg = "No socket could be created"
88.1745 + for res in info:
88.1746 + af, socktype, proto, canonname, sa = res
88.1747 + try:
88.1748 + self.bind(af, socktype, proto)
88.1749 + except socket.error:
88.1750 + if self.socket:
88.1751 + self.socket.close()
88.1752 + self.socket = None
88.1753 + continue
88.1754 + break
88.1755 + if not self.socket:
88.1756 + raise socket.error(msg)
88.1757 +
88.1758 + # Timeout so KeyboardInterrupt can be caught on Win32
88.1759 + self.socket.settimeout(1)
88.1760 + self.socket.listen(self.request_queue_size)
88.1761 +
88.1762 + # Create worker threads
88.1763 + self.requests.start()
88.1764 +
88.1765 + self.ready = True
88.1766 + self._start_time = time.time()
88.1767 + while self.ready:
88.1768 + self.tick()
88.1769 + if self.interrupt:
88.1770 + while self.interrupt is True:
88.1771 + # Wait for self.stop() to complete. See _set_interrupt.
88.1772 + time.sleep(0.1)
88.1773 + if self.interrupt:
88.1774 + raise self.interrupt
88.1775 +
88.1776 + def bind(self, family, type, proto=0):
88.1777 + """Create (or recreate) the actual socket object."""
88.1778 + self.socket = socket.socket(family, type, proto)
88.1779 + prevent_socket_inheritance(self.socket)
88.1780 + self.socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
88.1781 + if self.nodelay and not isinstance(self.bind_addr, str):
88.1782 + self.socket.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)
88.1783 +
88.1784 + if self.ssl_adapter is not None:
88.1785 + self.socket = self.ssl_adapter.bind(self.socket)
88.1786 +
88.1787 + # If listening on the IPV6 any address ('::' = IN6ADDR_ANY),
88.1788 + # activate dual-stack. See http://www.cherrypy.org/ticket/871.
88.1789 + if (hasattr(socket, 'AF_INET6') and family == socket.AF_INET6
88.1790 + and self.bind_addr[0] in ('::', '::0', '::0.0.0.0')):
88.1791 + try:
88.1792 + self.socket.setsockopt(socket.IPPROTO_IPV6, socket.IPV6_V6ONLY, 0)
88.1793 + except (AttributeError, socket.error):
88.1794 + # Apparently, the socket option is not available in
88.1795 + # this machine's TCP stack
88.1796 + pass
88.1797 +
88.1798 + self.socket.bind(self.bind_addr)
88.1799 +
88.1800 + def tick(self):
88.1801 + """Accept a new connection and put it on the Queue."""
88.1802 + try:
88.1803 + s, addr = self.socket.accept()
88.1804 + if self.stats['Enabled']:
88.1805 + self.stats['Accepts'] += 1
88.1806 + if not self.ready:
88.1807 + return
88.1808 +
88.1809 + prevent_socket_inheritance(s)
88.1810 + if hasattr(s, 'settimeout'):
88.1811 + s.settimeout(self.timeout)
88.1812 +
88.1813 + makefile = CP_fileobject
88.1814 + ssl_env = {}
88.1815 + # if ssl cert and key are set, we try to be a secure HTTP server
88.1816 + if self.ssl_adapter is not None:
88.1817 + try:
88.1818 + s, ssl_env = self.ssl_adapter.wrap(s)
88.1819 + except NoSSLError:
88.1820 + msg = ("The client sent a plain HTTP request, but "
88.1821 + "this server only speaks HTTPS on this port.")
88.1822 + buf = ["%s 400 Bad Request\r\n" % self.protocol,
88.1823 + "Content-Length: %s\r\n" % len(msg),
88.1824 + "Content-Type: text/plain\r\n\r\n",
88.1825 + msg]
88.1826 +
88.1827 + wfile = CP_fileobject(s, "wb", DEFAULT_BUFFER_SIZE)
88.1828 + try:
88.1829 + wfile.sendall("".join(buf))
88.1830 + except socket.error, x:
88.1831 + if x.args[0] not in socket_errors_to_ignore:
88.1832 + raise
88.1833 + return
88.1834 + if not s:
88.1835 + return
88.1836 + makefile = self.ssl_adapter.makefile
88.1837 + # Re-apply our timeout since we may have a new socket object
88.1838 + if hasattr(s, 'settimeout'):
88.1839 + s.settimeout(self.timeout)
88.1840 +
88.1841 + conn = self.ConnectionClass(self, s, makefile)
88.1842 +
88.1843 + if not isinstance(self.bind_addr, basestring):
88.1844 + # optional values
88.1845 + # Until we do DNS lookups, omit REMOTE_HOST
88.1846 + if addr is None: # sometimes this can happen
88.1847 + # figure out if AF_INET or AF_INET6.
88.1848 + if len(s.getsockname()) == 2:
88.1849 + # AF_INET
88.1850 + addr = ('0.0.0.0', 0)
88.1851 + else:
88.1852 + # AF_INET6
88.1853 + addr = ('::', 0)
88.1854 + conn.remote_addr = addr[0]
88.1855 + conn.remote_port = addr[1]
88.1856 +
88.1857 + conn.ssl_env = ssl_env
88.1858 +
88.1859 + self.requests.put(conn)
88.1860 + except socket.timeout:
88.1861 + # The only reason for the timeout in start() is so we can
88.1862 + # notice keyboard interrupts on Win32, which don't interrupt
88.1863 + # accept() by default
88.1864 + return
88.1865 + except socket.error, x:
88.1866 + if self.stats['Enabled']:
88.1867 + self.stats['Socket Errors'] += 1
88.1868 + if x.args[0] in socket_error_eintr:
88.1869 + # I *think* this is right. EINTR should occur when a signal
88.1870 + # is received during the accept() call; all docs say retry
88.1871 + # the call, and I *think* I'm reading it right that Python
88.1872 + # will then go ahead and poll for and handle the signal
88.1873 + # elsewhere. See http://www.cherrypy.org/ticket/707.
88.1874 + return
88.1875 + if x.args[0] in socket_errors_nonblocking:
88.1876 + # Just try again. See http://www.cherrypy.org/ticket/479.
88.1877 + return
88.1878 + if x.args[0] in socket_errors_to_ignore:
88.1879 + # Our socket was closed.
88.1880 + # See http://www.cherrypy.org/ticket/686.
88.1881 + return
88.1882 + raise
88.1883 +
88.1884 + def _get_interrupt(self):
88.1885 + return self._interrupt
88.1886 + def _set_interrupt(self, interrupt):
88.1887 + self._interrupt = True
88.1888 + self.stop()
88.1889 + self._interrupt = interrupt
88.1890 + interrupt = property(_get_interrupt, _set_interrupt,
88.1891 + doc="Set this to an Exception instance to "
88.1892 + "interrupt the server.")
88.1893 +
88.1894 + def stop(self):
88.1895 + """Gracefully shutdown a server that is serving forever."""
88.1896 + self.ready = False
88.1897 + if self._start_time is not None:
88.1898 + self._run_time += (time.time() - self._start_time)
88.1899 + self._start_time = None
88.1900 +
88.1901 + sock = getattr(self, "socket", None)
88.1902 + if sock:
88.1903 + if not isinstance(self.bind_addr, basestring):
88.1904 + # Touch our own socket to make accept() return immediately.
88.1905 + try:
88.1906 + host, port = sock.getsockname()[:2]
88.1907 + except socket.error, x:
88.1908 + if x.args[0] not in socket_errors_to_ignore:
88.1909 + # Changed to use error code and not message
88.1910 + # See http://www.cherrypy.org/ticket/860.
88.1911 + raise
88.1912 + else:
88.1913 + # Note that we're explicitly NOT using AI_PASSIVE,
88.1914 + # here, because we want an actual IP to touch.
88.1915 + # localhost won't work if we've bound to a public IP,
88.1916 + # but it will if we bound to '0.0.0.0' (INADDR_ANY).
88.1917 + for res in socket.getaddrinfo(host, port, socket.AF_UNSPEC,
88.1918 + socket.SOCK_STREAM):
88.1919 + af, socktype, proto, canonname, sa = res
88.1920 + s = None
88.1921 + try:
88.1922 + s = socket.socket(af, socktype, proto)
88.1923 + # See http://groups.google.com/group/cherrypy-users/
88.1924 + # browse_frm/thread/bbfe5eb39c904fe0
88.1925 + s.settimeout(1.0)
88.1926 + s.connect((host, port))
88.1927 + s.close()
88.1928 + except socket.error:
88.1929 + if s:
88.1930 + s.close()
88.1931 + if hasattr(sock, "close"):
88.1932 + sock.close()
88.1933 + self.socket = None
88.1934 +
88.1935 + self.requests.stop(self.shutdown_timeout)
88.1936 +
88.1937 +
88.1938 +class Gateway(object):
88.1939 +
88.1940 + def __init__(self, req):
88.1941 + self.req = req
88.1942 +
88.1943 + def respond(self):
88.1944 + raise NotImplemented
88.1945 +
88.1946 +
88.1947 +# These may either be wsgiserver.SSLAdapter subclasses or the string names
88.1948 +# of such classes (in which case they will be lazily loaded).
88.1949 +ssl_adapters = {
88.1950 + 'builtin': 'cherrypy.wsgiserver.ssl_builtin.BuiltinSSLAdapter',
88.1951 + 'pyopenssl': 'cherrypy.wsgiserver.ssl_pyopenssl.pyOpenSSLAdapter',
88.1952 + }
88.1953 +
88.1954 +def get_ssl_adapter_class(name='pyopenssl'):
88.1955 + adapter = ssl_adapters[name.lower()]
88.1956 + if isinstance(adapter, basestring):
88.1957 + last_dot = adapter.rfind(".")
88.1958 + attr_name = adapter[last_dot + 1:]
88.1959 + mod_path = adapter[:last_dot]
88.1960 +
88.1961 + try:
88.1962 + mod = sys.modules[mod_path]
88.1963 + if mod is None:
88.1964 + raise KeyError()
88.1965 + except KeyError:
88.1966 + # The last [''] is important.
88.1967 + mod = __import__(mod_path, globals(), locals(), [''])
88.1968 +
88.1969 + # Let an AttributeError propagate outward.
88.1970 + try:
88.1971 + adapter = getattr(mod, attr_name)
88.1972 + except AttributeError:
88.1973 + raise AttributeError("'%s' object has no attribute '%s'"
88.1974 + % (mod_path, attr_name))
88.1975 +
88.1976 + return adapter
88.1977 +
88.1978 +# -------------------------------- WSGI Stuff -------------------------------- #
88.1979 +
88.1980 +
88.1981 +class CherryPyWSGIServer(HTTPServer):
88.1982 +
88.1983 + wsgi_version = (1, 0)
88.1984 +
88.1985 + def __init__(self, bind_addr, wsgi_app, numthreads=10, server_name=None,
88.1986 + max=-1, request_queue_size=5, timeout=10, shutdown_timeout=5):
88.1987 + self.requests = ThreadPool(self, min=numthreads or 1, max=max)
88.1988 + self.wsgi_app = wsgi_app
88.1989 + self.gateway = wsgi_gateways[self.wsgi_version]
88.1990 +
88.1991 + self.bind_addr = bind_addr
88.1992 + if not server_name:
88.1993 + server_name = socket.gethostname()
88.1994 + self.server_name = server_name
88.1995 + self.request_queue_size = request_queue_size
88.1996 +
88.1997 + self.timeout = timeout
88.1998 + self.shutdown_timeout = shutdown_timeout
88.1999 + self.clear_stats()
88.2000 +
88.2001 + def _get_numthreads(self):
88.2002 + return self.requests.min
88.2003 + def _set_numthreads(self, value):
88.2004 + self.requests.min = value
88.2005 + numthreads = property(_get_numthreads, _set_numthreads)
88.2006 +
88.2007 +
88.2008 +class WSGIGateway(Gateway):
88.2009 +
88.2010 + def __init__(self, req):
88.2011 + self.req = req
88.2012 + self.started_response = False
88.2013 + self.env = self.get_environ()
88.2014 + self.remaining_bytes_out = None
88.2015 +
88.2016 + def get_environ(self):
88.2017 + """Return a new environ dict targeting the given wsgi.version"""
88.2018 + raise NotImplemented
88.2019 +
88.2020 + def respond(self):
88.2021 + response = self.req.server.wsgi_app(self.env, self.start_response)
88.2022 + try:
88.2023 + for chunk in response:
88.2024 + # "The start_response callable must not actually transmit
88.2025 + # the response headers. Instead, it must store them for the
88.2026 + # server or gateway to transmit only after the first
88.2027 + # iteration of the application return value that yields
88.2028 + # a NON-EMPTY string, or upon the application's first
88.2029 + # invocation of the write() callable." (PEP 333)
88.2030 + if chunk:
88.2031 + if isinstance(chunk, unicode):
88.2032 + chunk = chunk.encode('ISO-8859-1')
88.2033 + self.write(chunk)
88.2034 + finally:
88.2035 + if hasattr(response, "close"):
88.2036 + response.close()
88.2037 +
88.2038 + def start_response(self, status, headers, exc_info = None):
88.2039 + """WSGI callable to begin the HTTP response."""
88.2040 + # "The application may call start_response more than once,
88.2041 + # if and only if the exc_info argument is provided."
88.2042 + if self.started_response and not exc_info:
88.2043 + raise AssertionError("WSGI start_response called a second "
88.2044 + "time with no exc_info.")
88.2045 + self.started_response = True
88.2046 +
88.2047 + # "if exc_info is provided, and the HTTP headers have already been
88.2048 + # sent, start_response must raise an error, and should raise the
88.2049 + # exc_info tuple."
88.2050 + if self.req.sent_headers:
88.2051 + try:
88.2052 + raise exc_info[0], exc_info[1], exc_info[2]
88.2053 + finally:
88.2054 + exc_info = None
88.2055 +
88.2056 + self.req.status = status
88.2057 + for k, v in headers:
88.2058 + if not isinstance(k, str):
88.2059 + raise TypeError("WSGI response header key %r is not a byte string." % k)
88.2060 + if not isinstance(v, str):
88.2061 + raise TypeError("WSGI response header value %r is not a byte string." % v)
88.2062 + if k.lower() == 'content-length':
88.2063 + self.remaining_bytes_out = int(v)
88.2064 + self.req.outheaders.extend(headers)
88.2065 +
88.2066 + return self.write
88.2067 +
88.2068 + def write(self, chunk):
88.2069 + """WSGI callable to write unbuffered data to the client.
88.2070 +
88.2071 + This method is also used internally by start_response (to write
88.2072 + data from the iterable returned by the WSGI application).
88.2073 + """
88.2074 + if not self.started_response:
88.2075 + raise AssertionError("WSGI write called before start_response.")
88.2076 +
88.2077 + chunklen = len(chunk)
88.2078 + rbo = self.remaining_bytes_out
88.2079 + if rbo is not None and chunklen > rbo:
88.2080 + if not self.req.sent_headers:
88.2081 + # Whew. We can send a 500 to the client.
88.2082 + self.req.simple_response("500 Internal Server Error",
88.2083 + "The requested resource returned more bytes than the "
88.2084 + "declared Content-Length.")
88.2085 + else:
88.2086 + # Dang. We have probably already sent data. Truncate the chunk
88.2087 + # to fit (so the client doesn't hang) and raise an error later.
88.2088 + chunk = chunk[:rbo]
88.2089 +
88.2090 + if not self.req.sent_headers:
88.2091 + self.req.sent_headers = True
88.2092 + self.req.send_headers()
88.2093 +
88.2094 + self.req.write(chunk)
88.2095 +
88.2096 + if rbo is not None:
88.2097 + rbo -= chunklen
88.2098 + if rbo < 0:
88.2099 + raise ValueError(
88.2100 + "Response body exceeds the declared Content-Length.")
88.2101 +
88.2102 +
88.2103 +class WSGIGateway_10(WSGIGateway):
88.2104 +
88.2105 + def get_environ(self):
88.2106 + """Return a new environ dict targeting the given wsgi.version"""
88.2107 + req = self.req
88.2108 + env = {
88.2109 + # set a non-standard environ entry so the WSGI app can know what
88.2110 + # the *real* server protocol is (and what features to support).
88.2111 + # See http://www.faqs.org/rfcs/rfc2145.html.
88.2112 + 'ACTUAL_SERVER_PROTOCOL': req.server.protocol,
88.2113 + 'PATH_INFO': req.path,
88.2114 + 'QUERY_STRING': req.qs,
88.2115 + 'REMOTE_ADDR': req.conn.remote_addr or '',
88.2116 + 'REMOTE_PORT': str(req.conn.remote_port or ''),
88.2117 + 'REQUEST_METHOD': req.method,
88.2118 + 'REQUEST_URI': req.uri,
88.2119 + 'SCRIPT_NAME': '',
88.2120 + 'SERVER_NAME': req.server.server_name,
88.2121 + # Bah. "SERVER_PROTOCOL" is actually the REQUEST protocol.
88.2122 + 'SERVER_PROTOCOL': req.request_protocol,
88.2123 + 'SERVER_SOFTWARE': req.server.software,
88.2124 + 'wsgi.errors': sys.stderr,
88.2125 + 'wsgi.input': req.rfile,
88.2126 + 'wsgi.multiprocess': False,
88.2127 + 'wsgi.multithread': True,
88.2128 + 'wsgi.run_once': False,
88.2129 + 'wsgi.url_scheme': req.scheme,
88.2130 + 'wsgi.version': (1, 0),
88.2131 + }
88.2132 +
88.2133 + if isinstance(req.server.bind_addr, basestring):
88.2134 + # AF_UNIX. This isn't really allowed by WSGI, which doesn't
88.2135 + # address unix domain sockets. But it's better than nothing.
88.2136 + env["SERVER_PORT"] = ""
88.2137 + else:
88.2138 + env["SERVER_PORT"] = str(req.server.bind_addr[1])
88.2139 +
88.2140 + # Request headers
88.2141 + for k, v in req.inheaders.iteritems():
88.2142 + env["HTTP_" + k.upper().replace("-", "_")] = v
88.2143 +
88.2144 + # CONTENT_TYPE/CONTENT_LENGTH
88.2145 + ct = env.pop("HTTP_CONTENT_TYPE", None)
88.2146 + if ct is not None:
88.2147 + env["CONTENT_TYPE"] = ct
88.2148 + cl = env.pop("HTTP_CONTENT_LENGTH", None)
88.2149 + if cl is not None:
88.2150 + env["CONTENT_LENGTH"] = cl
88.2151 +
88.2152 + if req.conn.ssl_env:
88.2153 + env.update(req.conn.ssl_env)
88.2154 +
88.2155 + return env
88.2156 +
88.2157 +
88.2158 +class WSGIGateway_u0(WSGIGateway_10):
88.2159 +
88.2160 + def get_environ(self):
88.2161 + """Return a new environ dict targeting the given wsgi.version"""
88.2162 + req = self.req
88.2163 + env_10 = WSGIGateway_10.get_environ(self)
88.2164 + env = dict([(k.decode('ISO-8859-1'), v) for k, v in env_10.iteritems()])
88.2165 + env[u'wsgi.version'] = ('u', 0)
88.2166 +
88.2167 + # Request-URI
88.2168 + env.setdefault(u'wsgi.url_encoding', u'utf-8')
88.2169 + try:
88.2170 + for key in [u"PATH_INFO", u"SCRIPT_NAME", u"QUERY_STRING"]:
88.2171 + env[key] = env_10[str(key)].decode(env[u'wsgi.url_encoding'])
88.2172 + except UnicodeDecodeError:
88.2173 + # Fall back to latin 1 so apps can transcode if needed.
88.2174 + env[u'wsgi.url_encoding'] = u'ISO-8859-1'
88.2175 + for key in [u"PATH_INFO", u"SCRIPT_NAME", u"QUERY_STRING"]:
88.2176 + env[key] = env_10[str(key)].decode(env[u'wsgi.url_encoding'])
88.2177 +
88.2178 + for k, v in sorted(env.items()):
88.2179 + if isinstance(v, str) and k not in ('REQUEST_URI', 'wsgi.input'):
88.2180 + env[k] = v.decode('ISO-8859-1')
88.2181 +
88.2182 + return env
88.2183 +
88.2184 +wsgi_gateways = {
88.2185 + (1, 0): WSGIGateway_10,
88.2186 + ('u', 0): WSGIGateway_u0,
88.2187 +}
88.2188 +
88.2189 +class WSGIPathInfoDispatcher(object):
88.2190 + """A WSGI dispatcher for dispatch based on the PATH_INFO.
88.2191 +
88.2192 + apps: a dict or list of (path_prefix, app) pairs.
88.2193 + """
88.2194 +
88.2195 + def __init__(self, apps):
88.2196 + try:
88.2197 + apps = apps.items()
88.2198 + except AttributeError:
88.2199 + pass
88.2200 +
88.2201 + # Sort the apps by len(path), descending
88.2202 + apps.sort(cmp=lambda x,y: cmp(len(x[0]), len(y[0])))
88.2203 + apps.reverse()
88.2204 +
88.2205 + # The path_prefix strings must start, but not end, with a slash.
88.2206 + # Use "" instead of "/".
88.2207 + self.apps = [(p.rstrip("/"), a) for p, a in apps]
88.2208 +
88.2209 + def __call__(self, environ, start_response):
88.2210 + path = environ["PATH_INFO"] or "/"
88.2211 + for p, app in self.apps:
88.2212 + # The apps list should be sorted by length, descending.
88.2213 + if path.startswith(p + "/") or path == p:
88.2214 + environ = environ.copy()
88.2215 + environ["SCRIPT_NAME"] = environ["SCRIPT_NAME"] + p
88.2216 + environ["PATH_INFO"] = path[len(p):]
88.2217 + return app(environ, start_response)
88.2218 +
88.2219 + start_response('404 Not Found', [('Content-Type', 'text/plain'),
88.2220 + ('Content-Length', '0')])
88.2221 + return ['']
88.2222 +
89.1 --- /dev/null Thu Jan 01 00:00:00 1970 +0000
89.2 +++ b/OpenSecurity/install/web.py-0.37/web/wsgiserver/ssl_builtin.py Mon Dec 02 14:02:05 2013 +0100
89.3 @@ -0,0 +1,72 @@
89.4 +"""A library for integrating Python's builtin ``ssl`` library with CherryPy.
89.5 +
89.6 +The ssl module must be importable for SSL functionality.
89.7 +
89.8 +To use this module, set ``CherryPyWSGIServer.ssl_adapter`` to an instance of
89.9 +``BuiltinSSLAdapter``.
89.10 +"""
89.11 +
89.12 +try:
89.13 + import ssl
89.14 +except ImportError:
89.15 + ssl = None
89.16 +
89.17 +from cherrypy import wsgiserver
89.18 +
89.19 +
89.20 +class BuiltinSSLAdapter(wsgiserver.SSLAdapter):
89.21 + """A wrapper for integrating Python's builtin ssl module with CherryPy."""
89.22 +
89.23 + certificate = None
89.24 + """The filename of the server SSL certificate."""
89.25 +
89.26 + private_key = None
89.27 + """The filename of the server's private key file."""
89.28 +
89.29 + def __init__(self, certificate, private_key, certificate_chain=None):
89.30 + if ssl is None:
89.31 + raise ImportError("You must install the ssl module to use HTTPS.")
89.32 + self.certificate = certificate
89.33 + self.private_key = private_key
89.34 + self.certificate_chain = certificate_chain
89.35 +
89.36 + def bind(self, sock):
89.37 + """Wrap and return the given socket."""
89.38 + return sock
89.39 +
89.40 + def wrap(self, sock):
89.41 + """Wrap and return the given socket, plus WSGI environ entries."""
89.42 + try:
89.43 + s = ssl.wrap_socket(sock, do_handshake_on_connect=True,
89.44 + server_side=True, certfile=self.certificate,
89.45 + keyfile=self.private_key, ssl_version=ssl.PROTOCOL_SSLv23)
89.46 + except ssl.SSLError, e:
89.47 + if e.errno == ssl.SSL_ERROR_EOF:
89.48 + # This is almost certainly due to the cherrypy engine
89.49 + # 'pinging' the socket to assert it's connectable;
89.50 + # the 'ping' isn't SSL.
89.51 + return None, {}
89.52 + elif e.errno == ssl.SSL_ERROR_SSL:
89.53 + if e.args[1].endswith('http request'):
89.54 + # The client is speaking HTTP to an HTTPS server.
89.55 + raise wsgiserver.NoSSLError
89.56 + raise
89.57 + return s, self.get_environ(s)
89.58 +
89.59 + # TODO: fill this out more with mod ssl env
89.60 + def get_environ(self, sock):
89.61 + """Create WSGI environ entries to be merged into each request."""
89.62 + cipher = sock.cipher()
89.63 + ssl_environ = {
89.64 + "wsgi.url_scheme": "https",
89.65 + "HTTPS": "on",
89.66 + 'SSL_PROTOCOL': cipher[1],
89.67 + 'SSL_CIPHER': cipher[0]
89.68 +## SSL_VERSION_INTERFACE string The mod_ssl program version
89.69 +## SSL_VERSION_LIBRARY string The OpenSSL program version
89.70 + }
89.71 + return ssl_environ
89.72 +
89.73 + def makefile(self, sock, mode='r', bufsize=-1):
89.74 + return wsgiserver.CP_fileobject(sock, mode, bufsize)
89.75 +
90.1 --- /dev/null Thu Jan 01 00:00:00 1970 +0000
90.2 +++ b/OpenSecurity/install/web.py-0.37/web/wsgiserver/ssl_pyopenssl.py Mon Dec 02 14:02:05 2013 +0100
90.3 @@ -0,0 +1,256 @@
90.4 +"""A library for integrating pyOpenSSL with CherryPy.
90.5 +
90.6 +The OpenSSL module must be importable for SSL functionality.
90.7 +You can obtain it from http://pyopenssl.sourceforge.net/
90.8 +
90.9 +To use this module, set CherryPyWSGIServer.ssl_adapter to an instance of
90.10 +SSLAdapter. There are two ways to use SSL:
90.11 +
90.12 +Method One
90.13 +----------
90.14 +
90.15 + * ``ssl_adapter.context``: an instance of SSL.Context.
90.16 +
90.17 +If this is not None, it is assumed to be an SSL.Context instance,
90.18 +and will be passed to SSL.Connection on bind(). The developer is
90.19 +responsible for forming a valid Context object. This approach is
90.20 +to be preferred for more flexibility, e.g. if the cert and key are
90.21 +streams instead of files, or need decryption, or SSL.SSLv3_METHOD
90.22 +is desired instead of the default SSL.SSLv23_METHOD, etc. Consult
90.23 +the pyOpenSSL documentation for complete options.
90.24 +
90.25 +Method Two (shortcut)
90.26 +---------------------
90.27 +
90.28 + * ``ssl_adapter.certificate``: the filename of the server SSL certificate.
90.29 + * ``ssl_adapter.private_key``: the filename of the server's private key file.
90.30 +
90.31 +Both are None by default. If ssl_adapter.context is None, but .private_key
90.32 +and .certificate are both given and valid, they will be read, and the
90.33 +context will be automatically created from them.
90.34 +"""
90.35 +
90.36 +import socket
90.37 +import threading
90.38 +import time
90.39 +
90.40 +from cherrypy import wsgiserver
90.41 +
90.42 +try:
90.43 + from OpenSSL import SSL
90.44 + from OpenSSL import crypto
90.45 +except ImportError:
90.46 + SSL = None
90.47 +
90.48 +
90.49 +class SSL_fileobject(wsgiserver.CP_fileobject):
90.50 + """SSL file object attached to a socket object."""
90.51 +
90.52 + ssl_timeout = 3
90.53 + ssl_retry = .01
90.54 +
90.55 + def _safe_call(self, is_reader, call, *args, **kwargs):
90.56 + """Wrap the given call with SSL error-trapping.
90.57 +
90.58 + is_reader: if False EOF errors will be raised. If True, EOF errors
90.59 + will return "" (to emulate normal sockets).
90.60 + """
90.61 + start = time.time()
90.62 + while True:
90.63 + try:
90.64 + return call(*args, **kwargs)
90.65 + except SSL.WantReadError:
90.66 + # Sleep and try again. This is dangerous, because it means
90.67 + # the rest of the stack has no way of differentiating
90.68 + # between a "new handshake" error and "client dropped".
90.69 + # Note this isn't an endless loop: there's a timeout below.
90.70 + time.sleep(self.ssl_retry)
90.71 + except SSL.WantWriteError:
90.72 + time.sleep(self.ssl_retry)
90.73 + except SSL.SysCallError, e:
90.74 + if is_reader and e.args == (-1, 'Unexpected EOF'):
90.75 + return ""
90.76 +
90.77 + errnum = e.args[0]
90.78 + if is_reader and errnum in wsgiserver.socket_errors_to_ignore:
90.79 + return ""
90.80 + raise socket.error(errnum)
90.81 + except SSL.Error, e:
90.82 + if is_reader and e.args == (-1, 'Unexpected EOF'):
90.83 + return ""
90.84 +
90.85 + thirdarg = None
90.86 + try:
90.87 + thirdarg = e.args[0][0][2]
90.88 + except IndexError:
90.89 + pass
90.90 +
90.91 + if thirdarg == 'http request':
90.92 + # The client is talking HTTP to an HTTPS server.
90.93 + raise wsgiserver.NoSSLError()
90.94 +
90.95 + raise wsgiserver.FatalSSLAlert(*e.args)
90.96 + except:
90.97 + raise
90.98 +
90.99 + if time.time() - start > self.ssl_timeout:
90.100 + raise socket.timeout("timed out")
90.101 +
90.102 + def recv(self, *args, **kwargs):
90.103 + buf = []
90.104 + r = super(SSL_fileobject, self).recv
90.105 + while True:
90.106 + data = self._safe_call(True, r, *args, **kwargs)
90.107 + buf.append(data)
90.108 + p = self._sock.pending()
90.109 + if not p:
90.110 + return "".join(buf)
90.111 +
90.112 + def sendall(self, *args, **kwargs):
90.113 + return self._safe_call(False, super(SSL_fileobject, self).sendall,
90.114 + *args, **kwargs)
90.115 +
90.116 + def send(self, *args, **kwargs):
90.117 + return self._safe_call(False, super(SSL_fileobject, self).send,
90.118 + *args, **kwargs)
90.119 +
90.120 +
90.121 +class SSLConnection:
90.122 + """A thread-safe wrapper for an SSL.Connection.
90.123 +
90.124 + ``*args``: the arguments to create the wrapped ``SSL.Connection(*args)``.
90.125 + """
90.126 +
90.127 + def __init__(self, *args):
90.128 + self._ssl_conn = SSL.Connection(*args)
90.129 + self._lock = threading.RLock()
90.130 +
90.131 + for f in ('get_context', 'pending', 'send', 'write', 'recv', 'read',
90.132 + 'renegotiate', 'bind', 'listen', 'connect', 'accept',
90.133 + 'setblocking', 'fileno', 'close', 'get_cipher_list',
90.134 + 'getpeername', 'getsockname', 'getsockopt', 'setsockopt',
90.135 + 'makefile', 'get_app_data', 'set_app_data', 'state_string',
90.136 + 'sock_shutdown', 'get_peer_certificate', 'want_read',
90.137 + 'want_write', 'set_connect_state', 'set_accept_state',
90.138 + 'connect_ex', 'sendall', 'settimeout', 'gettimeout'):
90.139 + exec("""def %s(self, *args):
90.140 + self._lock.acquire()
90.141 + try:
90.142 + return self._ssl_conn.%s(*args)
90.143 + finally:
90.144 + self._lock.release()
90.145 +""" % (f, f))
90.146 +
90.147 + def shutdown(self, *args):
90.148 + self._lock.acquire()
90.149 + try:
90.150 + # pyOpenSSL.socket.shutdown takes no args
90.151 + return self._ssl_conn.shutdown()
90.152 + finally:
90.153 + self._lock.release()
90.154 +
90.155 +
90.156 +class pyOpenSSLAdapter(wsgiserver.SSLAdapter):
90.157 + """A wrapper for integrating pyOpenSSL with CherryPy."""
90.158 +
90.159 + context = None
90.160 + """An instance of SSL.Context."""
90.161 +
90.162 + certificate = None
90.163 + """The filename of the server SSL certificate."""
90.164 +
90.165 + private_key = None
90.166 + """The filename of the server's private key file."""
90.167 +
90.168 + certificate_chain = None
90.169 + """Optional. The filename of CA's intermediate certificate bundle.
90.170 +
90.171 + This is needed for cheaper "chained root" SSL certificates, and should be
90.172 + left as None if not required."""
90.173 +
90.174 + def __init__(self, certificate, private_key, certificate_chain=None):
90.175 + if SSL is None:
90.176 + raise ImportError("You must install pyOpenSSL to use HTTPS.")
90.177 +
90.178 + self.context = None
90.179 + self.certificate = certificate
90.180 + self.private_key = private_key
90.181 + self.certificate_chain = certificate_chain
90.182 + self._environ = None
90.183 +
90.184 + def bind(self, sock):
90.185 + """Wrap and return the given socket."""
90.186 + if self.context is None:
90.187 + self.context = self.get_context()
90.188 + conn = SSLConnection(self.context, sock)
90.189 + self._environ = self.get_environ()
90.190 + return conn
90.191 +
90.192 + def wrap(self, sock):
90.193 + """Wrap and return the given socket, plus WSGI environ entries."""
90.194 + return sock, self._environ.copy()
90.195 +
90.196 + def get_context(self):
90.197 + """Return an SSL.Context from self attributes."""
90.198 + # See http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/442473
90.199 + c = SSL.Context(SSL.SSLv23_METHOD)
90.200 + c.use_privatekey_file(self.private_key)
90.201 + if self.certificate_chain:
90.202 + c.load_verify_locations(self.certificate_chain)
90.203 + c.use_certificate_file(self.certificate)
90.204 + return c
90.205 +
90.206 + def get_environ(self):
90.207 + """Return WSGI environ entries to be merged into each request."""
90.208 + ssl_environ = {
90.209 + "HTTPS": "on",
90.210 + # pyOpenSSL doesn't provide access to any of these AFAICT
90.211 +## 'SSL_PROTOCOL': 'SSLv2',
90.212 +## SSL_CIPHER string The cipher specification name
90.213 +## SSL_VERSION_INTERFACE string The mod_ssl program version
90.214 +## SSL_VERSION_LIBRARY string The OpenSSL program version
90.215 + }
90.216 +
90.217 + if self.certificate:
90.218 + # Server certificate attributes
90.219 + cert = open(self.certificate, 'rb').read()
90.220 + cert = crypto.load_certificate(crypto.FILETYPE_PEM, cert)
90.221 + ssl_environ.update({
90.222 + 'SSL_SERVER_M_VERSION': cert.get_version(),
90.223 + 'SSL_SERVER_M_SERIAL': cert.get_serial_number(),
90.224 +## 'SSL_SERVER_V_START': Validity of server's certificate (start time),
90.225 +## 'SSL_SERVER_V_END': Validity of server's certificate (end time),
90.226 + })
90.227 +
90.228 + for prefix, dn in [("I", cert.get_issuer()),
90.229 + ("S", cert.get_subject())]:
90.230 + # X509Name objects don't seem to have a way to get the
90.231 + # complete DN string. Use str() and slice it instead,
90.232 + # because str(dn) == "<X509Name object '/C=US/ST=...'>"
90.233 + dnstr = str(dn)[18:-2]
90.234 +
90.235 + wsgikey = 'SSL_SERVER_%s_DN' % prefix
90.236 + ssl_environ[wsgikey] = dnstr
90.237 +
90.238 + # The DN should be of the form: /k1=v1/k2=v2, but we must allow
90.239 + # for any value to contain slashes itself (in a URL).
90.240 + while dnstr:
90.241 + pos = dnstr.rfind("=")
90.242 + dnstr, value = dnstr[:pos], dnstr[pos + 1:]
90.243 + pos = dnstr.rfind("/")
90.244 + dnstr, key = dnstr[:pos], dnstr[pos + 1:]
90.245 + if key and value:
90.246 + wsgikey = 'SSL_SERVER_%s_DN_%s' % (prefix, key)
90.247 + ssl_environ[wsgikey] = value
90.248 +
90.249 + return ssl_environ
90.250 +
90.251 + def makefile(self, sock, mode='r', bufsize=-1):
90.252 + if SSL and isinstance(sock, SSL.ConnectionType):
90.253 + timeout = sock.gettimeout()
90.254 + f = SSL_fileobject(sock, mode, bufsize)
90.255 + f.ssl_timeout = timeout
90.256 + return f
90.257 + else:
90.258 + return wsgiserver.CP_fileobject(sock, mode, bufsize)
90.259 +
91.1 --- /dev/null Thu Jan 01 00:00:00 1970 +0000
91.2 +++ b/OpenSecurity/server/opensecurityd.py Mon Dec 02 14:02:05 2013 +0100
91.3 @@ -0,0 +1,178 @@
91.4 +#!/bin/env python
91.5 +# -*- coding: utf-8 -*-
91.6 +
91.7 +# ------------------------------------------------------------
91.8 +# opensecurityd
91.9 +#
91.10 +# the opensecurityd as RESTful server
91.11 +#
91.12 +# Autor: Oliver Maurhart, <oliver.maurhart@ait.ac.at>
91.13 +#
91.14 +# Copyright (C) 2013 AIT Austrian Institute of Technology
91.15 +# AIT Austrian Institute of Technology GmbH
91.16 +# Donau-City-Strasse 1 | 1220 Vienna | Austria
91.17 +# http://www.ait.ac.at
91.18 +#
91.19 +# This program is free software; you can redistribute it and/or
91.20 +# modify it under the terms of the GNU General Public License
91.21 +# as published by the Free Software Foundation version 2.
91.22 +#
91.23 +# This program is distributed in the hope that it will be useful,
91.24 +# but WITHOUT ANY WARRANTY; without even the implied warranty of
91.25 +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
91.26 +# GNU General Public License for more details.
91.27 +#
91.28 +# You should have received a copy of the GNU General Public License
91.29 +# along with this program; if not, write to the Free Software
91.30 +# Foundation, Inc., 51 Franklin Street, Fifth Floor,
91.31 +# Boston, MA 02110-1301, USA.
91.32 +# ------------------------------------------------------------
91.33 +
91.34 +
91.35 +# ------------------------------------------------------------
91.36 +# imports
91.37 +
91.38 +import os
91.39 +import os.path
91.40 +import subprocess
91.41 +import sys
91.42 +import web
91.43 +
91.44 +# local
91.45 +from environment import Environment
91.46 +
91.47 +
91.48 +# ------------------------------------------------------------
91.49 +# const
91.50 +
91.51 +
91.52 +__version__ = "0.1"
91.53 +
91.54 +
91.55 +"""All the URLs we know mapping to class handler"""
91.56 +opensecurity_urls = (
91.57 + '/application', 'os_application',
91.58 + '/device', 'os_device',
91.59 + '/device/credentials', 'os_device_credentials',
91.60 + '/device/password', 'os_device_password',
91.61 + '/', 'os_root'
91.62 +)
91.63 +
91.64 +
91.65 +# ------------------------------------------------------------
91.66 +# code
91.67 +
91.68 +
91.69 +class os_application:
91.70 +
91.71 + """OpenSecurity '/application' handler.
91.72 +
91.73 + This is called on GET /application?vm=VM-ID&app=APP-ID
91.74 + This tries to access the vm identified with the label VM-ID
91.75 + and launched the application identified APP-ID
91.76 + """
91.77 +
91.78 + def GET(self):
91.79 +
91.80 + # pick the arguments
91.81 + args = web.input()
91.82 +
91.83 + # we _need_ a vm
91.84 + if not "vm" in args:
91.85 + raise web.badrequest()
91.86 +
91.87 + # we _need_ a app
91.88 + if not "app" in args:
91.89 + raise web.badrequest()
91.90 +
91.91 + ## TODO: HARD CODED STUFF HERE! THIS SHOULD BE FLEXIBLE!
91.92 + ssh_private_key = os.path.join(Environment("opensecurity").data_path, 'share', '192.168.56.15.ppk')
91.93 + putty_session = '192.168.56.15'
91.94 + process_command = ['plink.exe', '-i', ssh_private_key, putty_session, args.app]
91.95 + si = subprocess.STARTUPINFO()
91.96 + si.dwFlags = subprocess.STARTF_USESHOWWINDOW
91.97 + si.wShowWindow = subprocess.SW_HIDE
91.98 + print('tyring to launch: ' + ' '.join(process_command))
91.99 + process = subprocess.Popen(process_command, shell = True)
91.100 + return 'launched: ' + ' '.join(process_command)
91.101 +
91.102 +
91.103 +class os_device:
91.104 +
91.105 + """OpenSecurity '/device' handler"""
91.106 +
91.107 + def GET(self):
91.108 + return "os_device"
91.109 +
91.110 +
91.111 +class os_device_credentials:
91.112 +
91.113 + """OpenSecurity '/device/credentials' handler.
91.114 +
91.115 + This is called on GET /device/credentials?id=DEVICE-ID.
91.116 + Ideally this should pop up a user dialog to insert his
91.117 + credentials based the DEVICE-ID
91.118 + """
91.119 +
91.120 + def GET(self):
91.121 +
91.122 + # pick the arguments
91.123 + args = web.input()
91.124 +
91.125 + # we _need_ a device id
91.126 + if not "id" in args:
91.127 + raise web.badrequest()
91.128 +
91.129 + # invoke the user dialog as a subprocess
91.130 + dlg_credentials_image = os.path.join(sys.path[0], 'opensecurity-dialog.py')
91.131 + process_command = [sys.executable, dlg_credentials_image, 'credentials', 'Please provide credentials for accessing \ndevice: "{0}".'.format(args.id)]
91.132 + process = subprocess.Popen(process_command, shell = False, stdout = subprocess.PIPE)
91.133 + result = process.communicate()[0]
91.134 + if process.returncode != 0:
91.135 + return 'Credentials request has been aborted.'
91.136 +
91.137 + return result
91.138 +
91.139 +
91.140 +class os_device_password:
91.141 +
91.142 + """OpenSecurity '/device/password' handler.
91.143 +
91.144 + This is called on GET /device/password?id=DEVICE-ID.
91.145 + Ideally this should pop up a user dialog to insert his
91.146 + password based the DEVICE-ID
91.147 + """
91.148 +
91.149 + def GET(self):
91.150 +
91.151 + # pick the arguments
91.152 + args = web.input()
91.153 +
91.154 + # we _need_ a device id
91.155 + if not "id" in args:
91.156 + raise web.badrequest()
91.157 +
91.158 + # invoke the user dialog as a subprocess
91.159 + dlg_credentials_image = os.path.join(sys.path[0], 'opensecurity-dialog.py')
91.160 + process_command = [sys.executable, dlg_credentials_image, 'password', 'Please provide a password for accessing \ndevice: "{0}".'.format(args.id)]
91.161 + process = subprocess.Popen(process_command, shell = False, stdout = subprocess.PIPE)
91.162 + result = process.communicate()[0]
91.163 + if process.returncode != 0:
91.164 + return 'Credentials request has been aborted.'
91.165 +
91.166 + return result
91.167 +
91.168 +
91.169 +class os_root:
91.170 +
91.171 + """OpenSecurity '/' handler"""
91.172 +
91.173 + def GET(self):
91.174 + return "OpenSecurity-Server { \"version\": \"%s\" }" % __version__
91.175 +
91.176 +
91.177 +# start
91.178 +if __name__ == "__main__":
91.179 + server = web.application(opensecurity_urls, globals())
91.180 + server.run()
91.181 +
