[331] | 1 | ENGINE
|
---|
| 2 | ======
|
---|
| 3 |
|
---|
| 4 | With OpenSSL 0.9.6, a new component was added to support alternative
|
---|
| 5 | cryptography implementations, most commonly for interfacing with external
|
---|
| 6 | crypto devices (eg. accelerator cards). This component is called ENGINE,
|
---|
| 7 | and its presence in OpenSSL 0.9.6 (and subsequent bug-fix releases)
|
---|
| 8 | caused a little confusion as 0.9.6** releases were rolled in two
|
---|
| 9 | versions, a "standard" and an "engine" version. In development for 0.9.7,
|
---|
| 10 | the ENGINE code has been merged into the main branch and will be present
|
---|
| 11 | in the standard releases from 0.9.7 forwards.
|
---|
| 12 |
|
---|
| 13 | There are currently built-in ENGINE implementations for the following
|
---|
| 14 | crypto devices:
|
---|
| 15 |
|
---|
| 16 | o Cryptodev
|
---|
| 17 | o Microsoft CryptoAPI
|
---|
| 18 | o VIA Padlock
|
---|
| 19 | o nCipher CHIL
|
---|
| 20 |
|
---|
| 21 | In addition, dynamic binding to external ENGINE implementations is now
|
---|
| 22 | provided by a special ENGINE called "dynamic". See the "DYNAMIC ENGINE"
|
---|
| 23 | section below for details.
|
---|
| 24 |
|
---|
| 25 | At this stage, a number of things are still needed and are being worked on:
|
---|
| 26 |
|
---|
| 27 | 1 Integration of EVP support.
|
---|
| 28 | 2 Configuration support.
|
---|
| 29 | 3 Documentation!
|
---|
| 30 |
|
---|
| 31 | 1 With respect to EVP, this relates to support for ciphers and digests in
|
---|
| 32 | the ENGINE model so that alternative implementations of existing
|
---|
| 33 | algorithms/modes (or previously unimplemented ones) can be provided by
|
---|
| 34 | ENGINE implementations.
|
---|
| 35 |
|
---|
| 36 | 2 Configuration support currently exists in the ENGINE API itself, in the
|
---|
| 37 | form of "control commands". These allow an application to expose to the
|
---|
| 38 | user/admin the set of commands and parameter types a given ENGINE
|
---|
| 39 | implementation supports, and for an application to directly feed string
|
---|
| 40 | based input to those ENGINEs, in the form of name-value pairs. This is an
|
---|
| 41 | extensible way for ENGINEs to define their own "configuration" mechanisms
|
---|
| 42 | that are specific to a given ENGINE (eg. for a particular hardware
|
---|
| 43 | device) but that should be consistent across *all* OpenSSL-based
|
---|
| 44 | applications when they use that ENGINE. Work is in progress (or at least
|
---|
| 45 | in planning) for supporting these control commands from the CONF (or
|
---|
| 46 | NCONF) code so that applications using OpenSSL's existing configuration
|
---|
| 47 | file format can have ENGINE settings specified in much the same way.
|
---|
| 48 | Presently however, applications must use the ENGINE API itself to provide
|
---|
| 49 | such functionality. To see first hand the types of commands available
|
---|
| 50 | with the various compiled-in ENGINEs (see further down for dynamic
|
---|
| 51 | ENGINEs), use the "engine" openssl utility with full verbosity, ie;
|
---|
| 52 | openssl engine -vvvv
|
---|
| 53 |
|
---|
| 54 | 3 Documentation? Volunteers welcome! The source code is reasonably well
|
---|
| 55 | self-documenting, but some summaries and usage instructions are needed -
|
---|
| 56 | moreover, they are needed in the same POD format the existing OpenSSL
|
---|
| 57 | documentation is provided in. Any complete or incomplete contributions
|
---|
| 58 | would help make this happen.
|
---|
| 59 |
|
---|
| 60 | STABILITY & BUG-REPORTS
|
---|
| 61 | =======================
|
---|
| 62 |
|
---|
| 63 | What already exists is fairly stable as far as it has been tested, but
|
---|
| 64 | the test base has been a bit small most of the time. For the most part,
|
---|
| 65 | the vendors of the devices these ENGINEs support have contributed to the
|
---|
| 66 | development and/or testing of the implementations, and *usually* (with no
|
---|
| 67 | guarantees) have experience in using the ENGINE support to drive their
|
---|
| 68 | devices from common OpenSSL-based applications. Bugs and/or inexplicable
|
---|
| 69 | behaviour in using a specific ENGINE implementation should be sent to the
|
---|
| 70 | author of that implementation (if it is mentioned in the corresponding C
|
---|
| 71 | file), and in the case of implementations for commercial hardware
|
---|
| 72 | devices, also through whatever vendor support channels are available. If
|
---|
| 73 | none of this is possible, or the problem seems to be something about the
|
---|
| 74 | ENGINE API itself (ie. not necessarily specific to a particular ENGINE
|
---|
| 75 | implementation) then you should mail complete details to the relevant
|
---|
| 76 | OpenSSL mailing list. For a definition of "complete details", refer to
|
---|
| 77 | the OpenSSL "README" file. As for which list to send it to;
|
---|
| 78 |
|
---|
| 79 | openssl-users: if you are *using* the ENGINE abstraction, either in an
|
---|
| 80 | pre-compiled application or in your own application code.
|
---|
| 81 |
|
---|
| 82 | openssl-dev: if you are discussing problems with OpenSSL source code.
|
---|
| 83 |
|
---|
| 84 | USAGE
|
---|
| 85 | =====
|
---|
| 86 |
|
---|
| 87 | The default "openssl" ENGINE is always chosen when performing crypto
|
---|
| 88 | operations unless you specify otherwise. You must actively tell the
|
---|
| 89 | openssl utility commands to use anything else through a new command line
|
---|
| 90 | switch called "-engine". Also, if you want to use the ENGINE support in
|
---|
| 91 | your own code to do something similar, you must likewise explicitly
|
---|
| 92 | select the ENGINE implementation you want.
|
---|
| 93 |
|
---|
| 94 | Depending on the type of hardware, system, and configuration, "settings"
|
---|
| 95 | may need to be applied to an ENGINE for it to function as expected/hoped.
|
---|
| 96 | The recommended way of doing this is for the application to support
|
---|
| 97 | ENGINE "control commands" so that each ENGINE implementation can provide
|
---|
| 98 | whatever configuration primitives it might require and the application
|
---|
| 99 | can allow the user/admin (and thus the hardware vendor's support desk
|
---|
| 100 | also) to provide any such input directly to the ENGINE implementation.
|
---|
| 101 | This way, applications do not need to know anything specific to any
|
---|
| 102 | device, they only need to provide the means to carry such user/admin
|
---|
| 103 | input through to the ENGINE in question. Ie. this connects *you* (and
|
---|
| 104 | your helpdesk) to the specific ENGINE implementation (and device), and
|
---|
| 105 | allows application authors to not get buried in hassle supporting
|
---|
| 106 | arbitrary devices they know (and care) nothing about.
|
---|
| 107 |
|
---|
| 108 | A new "openssl" utility, "openssl engine", has been added in that allows
|
---|
| 109 | for testing and examination of ENGINE implementations. Basic usage
|
---|
| 110 | instructions are available by specifying the "-?" command line switch.
|
---|
| 111 |
|
---|
| 112 | DYNAMIC ENGINES
|
---|
| 113 | ===============
|
---|
| 114 |
|
---|
| 115 | The new "dynamic" ENGINE provides a low-overhead way to support ENGINE
|
---|
| 116 | implementations that aren't pre-compiled and linked into OpenSSL-based
|
---|
| 117 | applications. This could be because existing compiled-in implementations
|
---|
| 118 | have known problems and you wish to use a newer version with an existing
|
---|
| 119 | application. It could equally be because the application (or OpenSSL
|
---|
| 120 | library) you are using simply doesn't have support for the ENGINE you
|
---|
| 121 | wish to use, and the ENGINE provider (eg. hardware vendor) is providing
|
---|
| 122 | you with a self-contained implementation in the form of a shared-library.
|
---|
| 123 | The other use-case for "dynamic" is with applications that wish to
|
---|
| 124 | maintain the smallest foot-print possible and so do not link in various
|
---|
| 125 | ENGINE implementations from OpenSSL, but instead leaves you to provide
|
---|
| 126 | them, if you want them, in the form of "dynamic"-loadable
|
---|
| 127 | shared-libraries. It should be possible for hardware vendors to provide
|
---|
| 128 | their own shared-libraries to support arbitrary hardware to work with
|
---|
| 129 | applications based on OpenSSL 0.9.7 or later. If you're using an
|
---|
| 130 | application based on 0.9.7 (or later) and the support you desire is only
|
---|
| 131 | announced for versions later than the one you need, ask the vendor to
|
---|
| 132 | backport their ENGINE to the version you need.
|
---|
| 133 |
|
---|
| 134 | How does "dynamic" work?
|
---|
| 135 | ------------------------
|
---|
| 136 | The dynamic ENGINE has a special flag in its implementation such that
|
---|
| 137 | every time application code asks for the 'dynamic' ENGINE, it in fact
|
---|
| 138 | gets its own copy of it. As such, multi-threaded code (or code that
|
---|
| 139 | multiplexes multiple uses of 'dynamic' in a single application in any
|
---|
| 140 | way at all) does not get confused by 'dynamic' being used to do many
|
---|
| 141 | independent things. Other ENGINEs typically don't do this so there is
|
---|
| 142 | only ever 1 ENGINE structure of its type (and reference counts are used
|
---|
| 143 | to keep order). The dynamic ENGINE itself provides absolutely no
|
---|
| 144 | cryptographic functionality, and any attempt to "initialise" the ENGINE
|
---|
| 145 | automatically fails. All it does provide are a few "control commands"
|
---|
| 146 | that can be used to control how it will load an external ENGINE
|
---|
| 147 | implementation from a shared-library. To see these control commands,
|
---|
| 148 | use the command-line;
|
---|
| 149 |
|
---|
| 150 | openssl engine -vvvv dynamic
|
---|
| 151 |
|
---|
| 152 | The "SO_PATH" control command should be used to identify the
|
---|
| 153 | shared-library that contains the ENGINE implementation, and "NO_VCHECK"
|
---|
| 154 | might possibly be useful if there is a minor version conflict and you
|
---|
| 155 | (or a vendor helpdesk) is convinced you can safely ignore it.
|
---|
| 156 | "ID" is probably only needed if a shared-library implements
|
---|
| 157 | multiple ENGINEs, but if you know the engine id you expect to be using,
|
---|
| 158 | it doesn't hurt to specify it (and this provides a sanity check if
|
---|
| 159 | nothing else). "LIST_ADD" is only required if you actually wish the
|
---|
| 160 | loaded ENGINE to be discoverable by application code later on using the
|
---|
| 161 | ENGINE's "id". For most applications, this isn't necessary - but some
|
---|
| 162 | application authors may have nifty reasons for using it. The "LOAD"
|
---|
| 163 | command is the only one that takes no parameters and is the command
|
---|
| 164 | that uses the settings from any previous commands to actually *load*
|
---|
| 165 | the shared-library ENGINE implementation. If this command succeeds, the
|
---|
| 166 | (copy of the) 'dynamic' ENGINE will magically morph into the ENGINE
|
---|
| 167 | that has been loaded from the shared-library. As such, any control
|
---|
| 168 | commands supported by the loaded ENGINE could then be executed as per
|
---|
| 169 | normal. Eg. if ENGINE "foo" is implemented in the shared-library
|
---|
| 170 | "libfoo.so" and it supports some special control command "CMD_FOO", the
|
---|
| 171 | following code would load and use it (NB: obviously this code has no
|
---|
| 172 | error checking);
|
---|
| 173 |
|
---|
| 174 | ENGINE *e = ENGINE_by_id("dynamic");
|
---|
| 175 | ENGINE_ctrl_cmd_string(e, "SO_PATH", "/lib/libfoo.so", 0);
|
---|
| 176 | ENGINE_ctrl_cmd_string(e, "ID", "foo", 0);
|
---|
| 177 | ENGINE_ctrl_cmd_string(e, "LOAD", NULL, 0);
|
---|
| 178 | ENGINE_ctrl_cmd_string(e, "CMD_FOO", "some input data", 0);
|
---|
| 179 |
|
---|
| 180 | For testing, the "openssl engine" utility can be useful for this sort
|
---|
| 181 | of thing. For example the above code excerpt would achieve much the
|
---|
| 182 | same result as;
|
---|
| 183 |
|
---|
| 184 | openssl engine dynamic \
|
---|
| 185 | -pre SO_PATH:/lib/libfoo.so \
|
---|
| 186 | -pre ID:foo \
|
---|
| 187 | -pre LOAD \
|
---|
| 188 | -pre "CMD_FOO:some input data"
|
---|
| 189 |
|
---|
| 190 | Or to simply see the list of commands supported by the "foo" ENGINE;
|
---|
| 191 |
|
---|
| 192 | openssl engine -vvvv dynamic \
|
---|
| 193 | -pre SO_PATH:/lib/libfoo.so \
|
---|
| 194 | -pre ID:foo \
|
---|
| 195 | -pre LOAD
|
---|
| 196 |
|
---|
| 197 | Applications that support the ENGINE API and more specifically, the
|
---|
| 198 | "control commands" mechanism, will provide some way for you to pass
|
---|
| 199 | such commands through to ENGINEs. As such, you would select "dynamic"
|
---|
| 200 | as the ENGINE to use, and the parameters/commands you pass would
|
---|
| 201 | control the *actual* ENGINE used. Each command is actually a name-value
|
---|
| 202 | pair and the value can sometimes be omitted (eg. the "LOAD" command).
|
---|
| 203 | Whilst the syntax demonstrated in "openssl engine" uses a colon to
|
---|
| 204 | separate the command name from the value, applications may provide
|
---|
| 205 | their own syntax for making that separation (eg. a win32 registry
|
---|
| 206 | key-value pair may be used by some applications). The reason for the
|
---|
| 207 | "-pre" syntax in the "openssl engine" utility is that some commands
|
---|
| 208 | might be issued to an ENGINE *after* it has been initialised for use.
|
---|
| 209 | Eg. if an ENGINE implementation requires a smart-card to be inserted
|
---|
| 210 | during initialisation (or a PIN to be typed, or whatever), there may be
|
---|
| 211 | a control command you can issue afterwards to "forget" the smart-card
|
---|
| 212 | so that additional initialisation is no longer possible. In
|
---|
| 213 | applications such as web-servers, where potentially volatile code may
|
---|
| 214 | run on the same host system, this may provide some arguable security
|
---|
| 215 | value. In such a case, the command would be passed to the ENGINE after
|
---|
| 216 | it has been initialised for use, and so the "-post" switch would be
|
---|
| 217 | used instead. Applications may provide a different syntax for
|
---|
| 218 | supporting this distinction, and some may simply not provide it at all
|
---|
| 219 | ("-pre" is almost always what you're after, in reality).
|
---|
| 220 |
|
---|
| 221 | How do I build a "dynamic" ENGINE?
|
---|
| 222 | ----------------------------------
|
---|
| 223 | This question is trickier - currently OpenSSL bundles various ENGINE
|
---|
| 224 | implementations that are statically built in, and any application that
|
---|
| 225 | calls the "ENGINE_load_builtin_engines()" function will automatically
|
---|
| 226 | have all such ENGINEs available (and occupying memory). Applications
|
---|
| 227 | that don't call that function have no ENGINEs available like that and
|
---|
| 228 | would have to use "dynamic" to load any such ENGINE - but on the other
|
---|
| 229 | hand such applications would only have the memory footprint of any
|
---|
| 230 | ENGINEs explicitly loaded using user/admin provided control commands.
|
---|
| 231 | The main advantage of not statically linking ENGINEs and only using
|
---|
| 232 | "dynamic" for hardware support is that any installation using no
|
---|
| 233 | "external" ENGINE suffers no unnecessary memory footprint from unused
|
---|
| 234 | ENGINEs. Likewise, installations that do require an ENGINE incur the
|
---|
| 235 | overheads from only *that* ENGINE once it has been loaded.
|
---|
| 236 |
|
---|
| 237 | Sounds good? Maybe, but currently building an ENGINE implementation as
|
---|
| 238 | a shared-library that can be loaded by "dynamic" isn't automated in
|
---|
| 239 | OpenSSL's build process. It can be done manually quite easily however.
|
---|
| 240 | Such a shared-library can either be built with any OpenSSL code it
|
---|
| 241 | needs statically linked in, or it can link dynamically against OpenSSL
|
---|
| 242 | if OpenSSL itself is built as a shared library. The instructions are
|
---|
| 243 | the same in each case, but in the former (statically linked any
|
---|
| 244 | dependencies on OpenSSL) you must ensure OpenSSL is built with
|
---|
| 245 | position-independent code ("PIC"). The default OpenSSL compilation may
|
---|
| 246 | already specify the relevant flags to do this, but you should consult
|
---|
| 247 | with your compiler documentation if you are in any doubt.
|
---|
| 248 |
|
---|
| 249 | This example will show building the "atalla" ENGINE in the
|
---|
| 250 | crypto/engine/ directory as a shared-library for use via the "dynamic"
|
---|
| 251 | ENGINE.
|
---|
| 252 | 1) "cd" to the crypto/engine/ directory of a pre-compiled OpenSSL
|
---|
| 253 | source tree.
|
---|
| 254 | 2) Recompile at least one source file so you can see all the compiler
|
---|
| 255 | flags (and syntax) being used to build normally. Eg;
|
---|
| 256 | touch hw_atalla.c ; make
|
---|
| 257 | will rebuild "hw_atalla.o" using all such flags.
|
---|
| 258 | 3) Manually enter the same compilation line to compile the
|
---|
| 259 | "hw_atalla.c" file but with the following two changes;
|
---|
| 260 | (a) add "-DENGINE_DYNAMIC_SUPPORT" to the command line switches,
|
---|
| 261 | (b) change the output file from "hw_atalla.o" to something new,
|
---|
| 262 | eg. "tmp_atalla.o"
|
---|
| 263 | 4) Link "tmp_atalla.o" into a shared-library using the top-level
|
---|
| 264 | OpenSSL libraries to resolve any dependencies. The syntax for doing
|
---|
| 265 | this depends heavily on your system/compiler and is a nightmare
|
---|
| 266 | known well to anyone who has worked with shared-library portability
|
---|
| 267 | before. 'gcc' on Linux, for example, would use the following syntax;
|
---|
| 268 | gcc -shared -o dyn_atalla.so tmp_atalla.o -L../.. -lcrypto
|
---|
| 269 | 5) Test your shared library using "openssl engine" as explained in the
|
---|
| 270 | previous section. Eg. from the top-level directory, you might try;
|
---|
| 271 | apps/openssl engine -vvvv dynamic \
|
---|
| 272 | -pre SO_PATH:./crypto/engine/dyn_atalla.so -pre LOAD
|
---|
| 273 | If the shared-library loads successfully, you will see both "-pre"
|
---|
| 274 | commands marked as "SUCCESS" and the list of control commands
|
---|
| 275 | displayed (because of "-vvvv") will be the control commands for the
|
---|
| 276 | *atalla* ENGINE (ie. *not* the 'dynamic' ENGINE). You can also add
|
---|
| 277 | the "-t" switch to the utility if you want it to try and initialise
|
---|
| 278 | the atalla ENGINE for use to test any possible hardware/driver
|
---|
| 279 | issues.
|
---|
| 280 |
|
---|
| 281 | PROBLEMS
|
---|
| 282 | ========
|
---|
| 283 |
|
---|
| 284 | It seems like the ENGINE part doesn't work too well with CryptoSwift on Win32.
|
---|
| 285 | A quick test done right before the release showed that trying "openssl speed
|
---|
| 286 | -engine cswift" generated errors. If the DSO gets enabled, an attempt is made
|
---|
| 287 | to write at memory address 0x00000002.
|
---|
| 288 |
|
---|