List of Archived Posts

2006 Newsgroup Postings (05/31 - 06/15)

history of computing

history of computing

virtual memory

virtual memory

Google Architecture

virtual memory

Google Architecture

Google Architecture

Google Architecture

virtual memory

virtual memory

virtual memory

virtual memory

virtual memory

virtual memory

virtual memory

virtual memory

virtual memory

virtual memory

virtual memory

Why I use a Mac, anno 2006

Virtual Virtualizers

Virtual Virtualizers

Virtual Virtualizers

Google Architecture

Mainframe Linux Mythbusting (Was: Using Java in batch on z/OS?)

Google Architecture

Google Architecture

Google Architecture

Mainframe Linux Mythbusting (Was: Using Java in batch

One or two CPUs - the pros & cons

Google Architecture

Google Architecture

Google Architecture

Dual Core CPUs are slower than Dual Single core CPUs ??

Token-ring vs Ethernet - 10 years later

Token-ring vs Ethernet - 10 years later

Google Architecture

Token-ring vs Ethernet - 10 years later

Token-ring vs Ethernet - 10 years later

virtual memory

One or two CPUs - the pros & cons

The very first text editor

One or two CPUs - the pros & cons

The very first text editor

Mainframe Linux Mythbusting (Was: Using Java in batch on z/OS?)

Mainframe Linux Mythbusting (Was: Using Java in batch on z/OS?)

One or two CPUs - the pros & cons

virtual memory

Supercomputers Cost

Mainframe Linux Mythbusting (Was: Using Java in batch on

the new math: old battle of the sexes was: PDP-1

Mainframe Linux Mythbusting

Mainframe Linux Mythbusting (Was: Using Java in batch on z/OS?)

Memory Mapped I/O Vs I/O Mapped I/O

virtual memory

DEC's Hudson fab

DEC's Hudson fab

DEC's Hudson fab

Supercomputers Cost

DEC's Hudson fab

DEC's Hudson fab

Large Computer Rescue

DEC's Hudson fab

Large Computer Rescue

Why no double wide compare and swap on Sparc?

history of computing

From: lynn@garlic.com
Newsgroups: alt.folklore.computers
Subject: Re: history of computing
Date: Wed, 31 May 2006 06:48:46 -0700

Tim Shoppa wrote:

Relational databases for this sort of thing - where the truly
interesting stuff is by nature uncategorizable and the truly innovative
stuff erases all the category definitions - are a mistake.

I somehow currently believe that spatial databases in some very high
dimension are more useful for the concepts of "this inspired that" or
"this evolved at the same time as that but elsewhere" or "this and that
fed off each other" but even current spatial databases (a hot topic
over the past decade) fall down on the spongy multivariate relations
that I see.

We truly need a database system that is both spatial AND relational at
the same time to be useful.

The best approximation today is probably hypertext/Wiki-like with
humans drawing the links.

original relational/sql was system/r
http://www.garlic.com/~lynn/submain.html#systemr

at the same time i was doing some work for system/r ... i also got to
do some stuff for a more network oriented implementation ... that had
some of the same objectives as went into system/r implementation ...
i.e. abstracting physical pointers that were characterisitc of 60s
database implementation). thread discussing part of this issue:
http://www.garlic.com/~lynn/2005s.html#9 Flat Query
http://www.garlic.com/~lynn/2005s.html#12 Flat Query

a many times descendent of that work, i use for maintaining the
information that I utilize for generating the RFC index
http://www.garlic.com/~lynn/rfcietff.htm

and the various merged glossaries and taxonomies
http://www.garlic.com/~lynn/index.html#glosnote

in fact, i suspect that the high ratio of hrefs to actual text
accounts for the large number of daily hits from various search engine
implementations ... almost as if they are using it for daily test
case.

history of computing

From: lynn@garlic.com
Newsgroups: alt.folklore.computers
Subject: Re: history of computing
Date: Wed, 31 May 2006 08:24:04 -0700

Tim Shoppa wrote:

Clearly in making a post you can and do cross-reference back to
previous posts, but it would be remarkable (in a Philip K Dick inspired
moment) to figure out how to forward-reference to coming but not yet
written posts.

ref:
http://www.garlic.com/~lynn/2006l.html#0 history of computing

so the past posts get updated in the header with Refed field which are
the subsequent posts that reference the past posts; the RFC summary
information for references, obsoletes, updates, etc ... are done
similarly ... as are the merged taxonomy/glossary entries.

somewhat from a relational standpoint ... a side-effect of just
loading information results in a kind of full normalization, full
joining, and full indexing of the information just loaded ... along
with side-effect of all full join/index operations are always
bidirectional ... when there is a relationship from one-place to
another ... there is automatically always a reverse relationship from
the target back to the original.  it is also trivial to approx. a
RDBMS infrastructure (all primary index fields are unordered members
of set) and at the same time also represent a hierarchical ordering
(some members of the set may be precursor to other members of the same
set) and/or mesh ... (some members of a set have various kinds of
peer-to-peer relationships with other members of the same set).

it is difficult and/or somewhat contrived to approx. the full richness
of the infrastructure in HTML or even in a purely relational/sql
implementation.
http://www.garlic.com/~lynn/index.html

virtual memory

From: lynn@garlic.com
Newsgroups: comp.arch,alt.folklore.computers
Subject: Re: virtual memory
Date: Wed, 31 May 2006 08:56:51 -0700

robertwessel2@yahoo.com wrote:

Even the current 64 bit machines support pagein/pageout, but I don't
think (64 bit) zOS uses it for anything.  zVM supports it for 31 bit
guests, of course.

re:
http://www.garlic.com/~lynn/2006k.html#57 virtual memory

the earlier 3033 had a hack on this in the late 70s related to 24-bit
addressing.  instructions were 24-bit both real addressing and virtual
address.

i've commented before that the 3033s were feeling some pressure from
4341 clusters and the 16mbyte real memory address limit .... i.e. you
configure a six 4341 cluster, each 1mip (6mips aggregate), each
16mbyte real (96mbytes aggregate), each having six i/o channels (36
i/o channels aggregate) for about the same cost as a 4.5mip, 16 i/o
channel, 16mbyte 3033.

the 370 PTE (page table entry) had two undefined bits ... the standard
PTE was 12bit real 4k page numbers (i.e. 12bit*12bit ... gives 24bit,
16mbyte addressing). IDALS actually had field for doing i/o with real
address up to 31bits. so 3033 came out with 32mbyte real support ...
and used one of the unused PTE bits to allow defining 13bit (8192) 4k
real pages. IDALs could do i/o moving pages to/from disk ... above the
16mbyte line.

however, there was some number of things that required real addressing
of contents of virtual pages ... and for this there was a game played
with page table entries .... that had a virtual page point to a real
page below the 16mbyte line and a virtual page above the 16mbyte line
... and would copy the contents of the virtual page above the line
to the virtual page below the line.

misc. past posts mentioning the 3033 hack for 32mbyte real storage:
http://www.garlic.com/~lynn/2001m.html#15 departmental servers
http://www.garlic.com/~lynn/2002d.html#51 Hardest Mistake in Comp Arch to Fix
http://www.garlic.com/~lynn/2004g.html#20 Infiniband - practicalities for small clusters
http://www.garlic.com/~lynn/2004n.html#50 Integer types for 128-bit addressing
http://www.garlic.com/~lynn/2005.html#34 increasing addressable memory via paged memory?
http://www.garlic.com/~lynn/2005.html#43 increasing addressable memory via paged memory?
http://www.garlic.com/~lynn/2005m.html#28 IBM's mini computers--lack thereof
http://www.garlic.com/~lynn/2005p.html#1 Intel engineer discusses their dual-core design
http://www.garlic.com/~lynn/2005p.html#19 address space
http://www.garlic.com/~lynn/2005q.html#30 HASP/ASP JES/JES2/JES3
http://www.garlic.com/~lynn/2005u.html#44 POWER6 on zSeries?
http://www.garlic.com/~lynn/2006b.html#34 Multiple address spaces
http://www.garlic.com/~lynn/2006c.html#8 IBM 610 workstation computer
http://www.garlic.com/~lynn/2006e.html#45 using 3390 mod-9s

virtual memory

From: Anne & Lynn Wheeler <lynn@garlic.com>
Subject: Re: virtual memory
Newsgroups: comp.arch,alt.folklore.computers
Date: Wed, 31 May 2006 21:33:08 -0600

"Eric P." writes:

I found such studies, which appear to show that VMS's mechanism
is within, by looking at a graph, about 0.5% of LRU.

presumably this refers to "local" LRU as opposed to global/system
LRU. what was their basis for "LRU" ... was it at the per instruction
reference (updating the ordering of all pages as to the least recently
used at a per instruction referenced level) ... or was it at a larger
granularity?

one of the things that the several emulators did at the science center
http://www.garlic.com/~lynn/subtopic.html#545tech

was full instruction traces for exact storage reference sequence (at
the instruction level) and then compared LRU against things like
(belady's) OPT as well as the various implementations approx. LRU
... and then looked at global system thruput of the different
mechanisms.

part of this contributed to the analytical model that grew into
the sales/marketing assist tool available on the hone system
http://www.garlic.com/~lynn/subtopic.html#hone

called the performance predictor.

one of the issues was that standard (global or local) LRU degenerated
to FIFO under various conditions and much worse than RANDOM and/or
OPT. part of this was the slight of hand coding that i did in the
global LRU approximated that would degenerate to RANDOM in cases when
standard LRU degenerated to FIFO (and overall showed a 5-10 percent
improvement over straight LRU).
http://www.garlic.com/~lynn/subtopic.html#wsclock

the other issue in various of these studies wasn't so much how close
things got to LRU (or belady's OPT) but comparing overall system
thruput using different strategies.

leading up to releasing my resource manager on 11may76
http://www.garlic.com/~lynn/subtopic.html#fairshare

we did thousands of simulations and benchmarks ... that helped
calibrate both the stuff in the performance predictor as well in the
resource manager. we then did a varioation on the performance
predictor as well as a lot of synthetic benchmarks calibrated against
workload profiles from hundreds of systems, along with an automated
benchmarking infrastructure. the modified performance predictor got to
choose the workload profile, configuration, and system parameters for
the next benchmark, it then predicted what should happen, the
benchmark was run, and the measured results compared against the
predicted. the modified performance predictor updated the most recent
results ... and then used all the past benchmarks results plus the
most recent to select the next benchmark. leading up to the final
release of the resource manager ... 2000 such benchmarks were run
taking three months elapsed time.
http://www.garlic.com/~lynn/submain.html#bench

--
Anne & Lynn Wheeler | http://www.garlic.com/~lynn/

Google Architecture

From: Anne & Lynn Wheeler <lynn@garlic.com>
Subject: Re: Google Architecture
Newsgroups: bit.listserv.ibm-main,alt.folklore.computers
Date: Thu, 01 Jun 2006 10:43:19 -0600

Bill Richter wrote:

It appears that Google architecture is the antithesis of conventional
mainframe application achitecture in all aspects.

http://labs.google.com/papers/googlecluster-ieee.pdf

and the difference between that and loosely-coupled or parallel sysplex?

long ago and far away, my wife was con'ed to going to POK to be in
charge of loosely-coupled architecture ... she was in the same
organization with the guy in charge of tightly-coupled architecture.
while she had come up with peer-coupled shared data architecture
http://www.garlic.com/~lynn/submain.html#shareddata

it was tough slogging because all the attention was focused on
tightly-coupled architecture at the time. also she had battles with the
sna forces ... who wanted control of all communication that left the
processor complex (i.e. outside of direct disk i/o, etc).

part of the problem was that in the early days of SNA ... she had
co-authored a "peer-to-peer" network architecture with Bert Moldow ...
AWP39 (somewhat viewed in competition with sna). while SNA was tailored
for centralized control of a large number of dumb terminals ... it was
decidedly lacking in doing peer-to-peer operations with large numbers of
intelligent peers.

a trivial example was sjr had done cluster 4341 implementation used
highly optimized peer-to-peer protocols running over a slightly modified
trotter/3088 (i.e. eventually came out as conventional ctca ... but with
interconnection for eight processors/channels). peer-to-peer,
asynchronous could achieve cluster synchronization in under a second
elapsed time (for eight processors). doing the same thing with SNA
increased the elapsed time to approx. a minute. the group was forced to
only release the SNA-based implementation to customers ... which
obviously had severe scaling properties as the numbers in a cluster
increased.

recent cross-over reference mentioning 4341 clusters
http://www.garlic.com/~lynn/2006l.html#2 virtual memory

the communication division did help with significant uptake of PCs in
the commercial environment. a customer could replace a dumb 327x with a
PC for approx. the same price, get datacenter terminal emulation
connectivity and in the same desktop footprint also have some local
computing capability. as a result, you also found the communication
group with a large install base of products in the terminal emulation
market segment (with tens of millions of emulated dumb terminals)
http://www.garlic.com/~lynn/subnetwork.html#emulation

in the late 80s, we had come up with 3-tier architecture (as an
extension to 2-tier, client/server) and were out pitching it to customer
executives. however, the communication group had come up with SAA which
was oriented trying to stem the tide moving to peer-to-peer networking,
client/server, and away from dumb terminals. as a result, we tended to
take a lot of heat from the SAA forces.
http://www.garlic.com/~lynn/subnetwork.html#3tier

in the same time frame, a senior engineer from the disk group in san
jose managed to sneek a talk into the internal, annual world-wide
communication conference. he began his talk with the statement that the
communication group was going to be responsible for the demise of the
disk division. basically the disk division had been coming up with all
sorts of high-thruput, peer-to-peer network capability for PCs and
workstations to access the datacenter mainframe disk farms. the
communication was constantly opposing the efforts, protecting the
installed base of terminal emulation products. recent reference to
that talk:
http://www.garlic.com/~lynn/2006k.html#25 Can anythink kill x86-64?

i had started the high-speed data transport project in the early 80s ...
hsdt
http://www.garlic.com/~lynn/subnetwork.html#hsdt

and had a number of T1 (1.5mbit) and higher speed links for various
high-speed backbone applications. one friday, somebody in the
communication group started an internal discussion on high-speed
communication with some definitions ... recent posting referencing this
http://www.garlic.com/~lynn/2006e.html#36
definitions from communication group

       low-speed               <9.6kbits
       medium-speed            19.2kbits
       high-speed              56kbits
       very high-speed         1.5mbits

the following monday, i was in the far-east talking about purchasing
some hardware and they had the following definitions on their conference
room wall

       low-speed               <20mbits
       medium-speed            100mbits
       high-speed              200-300mbits
       very high-speed         >600mbits

part of this was the communication division 37xx product line only
supported up to 56kbit links. They had recently done a study to
determine if T1 support was required ... which concluded that in 8-10
years there would only be 200 mainframe customers requiring T1
communication support. The issue could have been that the people doing
the study were suppose to come up with the results supporting the
current product line ... or maybe they didn't understand the evolving
communication market segment, or possibly both.

their methodology was to look at customers using 37xx "fat pipes" ...
basically being able to operate multiple parallel 56kbit links as a
simulated single connection. They found several customers with two
parallel links, some with three parallel links, a few with four parallel
links and none with higher number. Based on that, they projected that it
would take nearly a decade before there were any number of customers
with parallel links approaching T1 (1.5.mbits) capacity.

the problem with the analysis at the time was that the telcos were
tariffing T1 at approx. the same as five 56kbit links. customers going
to more than four 56kbit links were buying full T1 and operating them
with hardware from other vendors. A trivial two-week survey turned up
200 mainframe customers with full T1 operations ... something that the
communication group was projecting wouldn't occur for nearly another decade.

so last fall, i was at a conference and there was a talk about "google
as a supercomputer". the basic thesis was that google was managing to
aggregate large collection of processing power and data storage well
into the supercomputer range ... and doing it for 1/3rd the cost of
the next closest implementation (in terms of cost).

slightly related old post
http://www.garlic.com/~lynn/95.html#13

from when we were working on scaling for our ha/cmp product
http://www.garlic.com/~lynn/subtopic.html#hacmp

virtual memory

From: Anne & Lynn Wheeler <lynn@garlic.com>
Subject: Re: virtual memory
Newsgroups: comp.arch,alt.folklore.computers
Date: Thu, 01 Jun 2006 12:02:34 -0600

"Eric P." writes:

The article I'm looking at is not that specific, and I do not have
access to the ACM see what the cited papers say.

I suspect they mean True LRU. As the local working set size shrinks,
it will toss out and then quickly fault back (soft fault) pages.
Therefore as the working set size shrinks, the remaining elements
asymptote to LRU. A working set of 1 page will be the LRU page.

I am also interested in seeing the cite list for those analysis
papers.  I note that the dates on the papers are all 2 years after
VMS was first sold (1979) and 7 years after its design was started
(1975).  Presumably they had something more than a hope and a prayer
to go on when they came up with their approach.

under constrained resources, LRU replacement may even have more
predisposition towards FIFO ... i.e. the applications only have had
opportunity to touch a series of pages in sequence ... and there is
little or no difference between LRU replacement and FIFO replacement
(since the first page in is also the least recently referenced page).
It is frequently only in less constrained environment where the
application has had opportunity to have repeated references ... and
that the LRU reference pattern starts to diverge from FIFO.

it was in these particular circumstances (where LRU replacement
degenerated to equivalent to FIFO replacement) that RANDOM replacement
can be significantly better than LRU/FIFO replacement.

--
Anne & Lynn Wheeler | http://www.garlic.com/~lynn/

Google Architecture

From: Anne & Lynn Wheeler <lynn@garlic.com>
Subject: Re: Google Architecture
Newsgroups: bit.listserv.ibm-main,alt.folklore.computers
Date: Thu, 01 Jun 2006 12:33:10 -0600

Chris.Craddock@CA.COM (Craddock, Chris) writes:

GOOGLE is certainly a loosely coupled architecture, but as you of all
people would know, there are significant differences between that and a
parallel sysplex. The main feature they (and Amazon as well btw) focus
on is the full burdened price of their computational units including
power, cooling, footprint etc. and that makes economic sense for them
given the nature of their business application.

re:
http://www.garlic.com/~lynn/2006l.html#4 Google Architecture

so the issue is effectively how fast fault isolation/recovery/tolerant
technology becomes commodized. this is somewhat the scenario that
happened with RAID ... when they first appeared, they were frequently
depreciated compared to "mainframe" DASD ... but since then, they've
effectively turned into the standard.

for a little drift, i've repeated several times before what I did for
i/o supervisor for the dasd engineering and product test labs
(bldg. 14 & bldg 15)
http://www.garlic.com/~lynn/subtopic.html#disk

they had "testcells" ... basically hardware under development ...  the
term testcells somewhat come from the security provisions ...  the
test hardware were in individual heavy steel mesh "cages" (testing
cells) ... inside a secured machine room.

they had tried doing testing in an operating system environment
... but at the time, MVS had a MTBF of 15 mins operating with a single
testcell. i undertook to rewrite the i/o supervisor so that it would
never fail ... even when operating half-dozen to a dozen testcells
concurrently .... allowing the processor complex then to also be used
for some number of other tasks concurrently.

bldg 14/15 tended to get early engineering models of processors ...
also as part of disk testing. however, in the "stand-alone" mode of
operation ... the processors were dedicated to scheduled i/o testing
(which tended to be less than one percent cpu utilization).

with the bullet proof i/o supervisor ... the idle cpu could be put to
other tasks. at one point, bldg. 15 got the first engineering 3033
(outside of POK) dedicated for disk i/o testing. however, once we had
testing going on in an operating system environment, we could take
advantage of essentially, an otherwise idle processor.

one of the applications that we moved onto the machine was the air
bearing modeling that was going on as part of the development of the
3380 floating heads. SJR had a 370/195 that was being operated as an
internal service ... and the air bearing modeling might get an hour or
so a couple times a month. however, with essentially an idle 3033
sitting across the street ... we could drastically improve that (the
370/195 was peak rated around 10mips ... but most codes ran in the
5mip range ... and the 3033 was in the 4.5mip range ... but
essentially unlimited amounts of 3033 time was still better than a hr
or so of 370/195 time a couple times a month).

--
Anne & Lynn Wheeler | http://www.garlic.com/~lynn/

Google Architecture

From: Anne & Lynn Wheeler <lynn@garlic.com>
Subject: Re: Google Architecture
Newsgroups: bit.listserv.ibm-main,alt.folklore.computers
Date: Thu, 01 Jun 2006 20:17:52 -0600

rkuebbin writes:

Acutally financial query might use this architecture, except that the
significant difference between this workload and "typical commercial"
workloads -- they do no updates.  Therefore no data integrity issues.

re:
http://www.garlic.com/~lynn/2006l.html#4 Google Architecture
http://www.garlic.com/~lynn/2006l.html#6 Google Architecture

we took some amount of heat in the 80s from the communication group
working on high-speed data transport
http://www.garlic.com/~lynn/subnetwork.html#hsdt
and 3-tier architecture (as extension of 2-tier, client/server)
http://www.garlic.com/~lynn/subnetwork.html#3tier

then in the early 90s ... when we were working on scaling
non-mainframe loosely-coupled for the commercial market
http://www.garlic.com/~lynn/subtopic.html#hacmp

we got hit and told we couldn't work on anything involving
more than four processors ... minor reference:
http://www.garlic.com/~lynn/95.html#13

however, the cluster scaling has evolved in a number of ways.
high-energy physics picked it up and evolved it as something called
GRID. a number of vendors also contributed a lot of work on GRID
technology and since are out pushing it in various commercial market
segments ... including financial. some of the early financial adopters
are using GRID for doing complex financial analysis in real-time.

some topic drift ... i gave a talk a couple years ago at the
global grid forum
http://forge.ggf.org/sf/docman/do/listDocuments/projects.ogsa-wg/docman.root.meeting_materials_and_minutes.ggf_meetings.ggf11

.... select GGF11-design-security-nakedkey in the above.

misc. GRID related news article in the commercial market

Investment Banks Using Grid Computing Models
http://www.epaynews.com/index.cgi?survey=&ref=browse&f=view&id=1148478974861413176&block=
ASPEED Taking Financial Grids to the Next Level
http://www.gridtoday.com/grid/673718.html
Wachovia uses grid technology to speed up transaction apps
http://www.computerworld.com/action/article.do?command=viewArticleBasic&articleId=9000476
Grid Computing That Heals Itself
http://www.enterpriseitplanet.com/networking/news/article.php/3606041
GRID WORLD 2006: New IBM software brings autonomic computing to Grids
http://www.enterprisenetworksandservers.com/newsflash/art.php?589

as somewhat referenced in a couple of the above ("batch processing
going back 50 years")... bringing "batch" to GRID can be somewhat
viewed as JES3 on steroids. before getting con'ed into going to pok to
be in charge of loosely coupled architecture
http://www.garlic.com/~lynn/submain.html#shareddata

my wife had been in the JES group in g'burg. She had been one of the
catchers for ASP ... as part of its transformation into JES3. She had
also done a business analysis of the major JES2 and JES3 features as
part of proposal for creating a merged product.  however, that never
made it very far ... in part because of a lot of internal politics.

random past posts mentioning jes3:
http://www.garlic.com/~lynn/99.html#58 When did IBM go object only
http://www.garlic.com/~lynn/99.html#92 MVS vs HASP vs JES (was 2821)
http://www.garlic.com/~lynn/99.html#209 Core (word usage) was anti-equipment etc
http://www.garlic.com/~lynn/2000.html#13 Computer of the century
http://www.garlic.com/~lynn/2000.html#76 Mainframe operating systems
http://www.garlic.com/~lynn/2000.html#78 Mainframe operating systems
http://www.garlic.com/~lynn/2000f.html#30 OT?
http://www.garlic.com/~lynn/2000f.html#37 OT?
http://www.garlic.com/~lynn/2001b.html#73 7090 vs. 7094 etc.
http://www.garlic.com/~lynn/2001c.html#69 Wheeler and Wheeler
http://www.garlic.com/~lynn/2001g.html#44 The Alpha/IA64 Hybrid
http://www.garlic.com/~lynn/2001g.html#46 The Alpha/IA64 Hybrid
http://www.garlic.com/~lynn/2001g.html#48 The Alpha/IA64 Hybrid
http://www.garlic.com/~lynn/2001n.html#11 OCO
http://www.garlic.com/~lynn/2002e.html#25 Crazy idea: has it been done?
http://www.garlic.com/~lynn/2002k.html#48 MVS 3.8J and NJE via CTC
http://www.garlic.com/~lynn/2002n.html#58 IBM S/370-168, 195, and 3033
http://www.garlic.com/~lynn/2002q.html#31 Collating on the S/360-2540 card reader?
http://www.garlic.com/~lynn/2002q.html#35 HASP:
http://www.garlic.com/~lynn/2004b.html#53 origin of the UNIX dd command
http://www.garlic.com/~lynn/2004c.html#6 If the x86 ISA could be redone
http://www.garlic.com/~lynn/2004e.html#51 Infiniband - practicalities for small clusters
http://www.garlic.com/~lynn/2004g.html#39 spool
http://www.garlic.com/~lynn/2004o.html#32 What system Release do you use... OS390? z/os? I'm a Vendor S
http://www.garlic.com/~lynn/2005o.html#39 JES unification project
http://www.garlic.com/~lynn/2005p.html#44 hasp, jes, rasp, aspen, gold
http://www.garlic.com/~lynn/2005p.html#45 HASP/ASP JES/JES2/JES3
http://www.garlic.com/~lynn/2005q.html#0 HASP/ASP JES/JES2/JES3
http://www.garlic.com/~lynn/2005q.html#7 HASP/ASP JES/JES2/JES3
http://www.garlic.com/~lynn/2005q.html#15 HASP/ASP JES/JES2/JES3
http://www.garlic.com/~lynn/2005q.html#16 HASP/ASP JES/JES2/JES3
http://www.garlic.com/~lynn/2005q.html#19 HASP/ASP JES/JES2/JES3
http://www.garlic.com/~lynn/2005q.html#30 HASP/ASP JES/JES2/JES3
http://www.garlic.com/~lynn/2005q.html#32 HASP/ASP JES/JES2/JES3
http://www.garlic.com/~lynn/2005s.html#16 Is a Hurricane about to hit IBM ?
http://www.garlic.com/~lynn/2006f.html#19 Over my head in a JES exit

--
Anne & Lynn Wheeler | http://www.garlic.com/~lynn/

Google Architecture

From: Anne & Lynn Wheeler <lynn@garlic.com>
Subject: Re: Google Architecture
Newsgroups: bit.listserv.ibm-main,alt.folklore.computers
Date: Fri, 02 Jun 2006 05:56:04 -0600

Anne & Lynn Wheeler wrote:

however, the cluster scaling has evolved in a number of ways.
high-energy physics picked it up and evolved it as something called
GRID. a number of vendors also contributed a lot of work on GRID
technology and since are out pushing it in various commercial market
segments ... including financial. some of the early financial adopters
are using GRID for doing complex financial analysis in real-time.

re:
http://www.garlic.com/~lynn/2006l.html#4 Google Architecture
http://www.garlic.com/~lynn/2006l.html#6 Google Architecture
http://www.garlic.com/~lynn/2006l.html#7 Google Architecture

recent news article from yesterday

Cern seeks to tighten security for data grid
http://www.vnunet.com/computing/news/2157258/cern-seeks-tighten-security

from above:

Although large data grids are only starting to be used in business, Cern
is seeing a lot of interest from industry. The lab is developing grids
that will reach across organisational boundaries, allowing multiple
institutions to share resources.

'Businesses are now becoming interested in this kind of grid,' said
Grey. 'Its use could enable suppliers and companies to share resources
and large corporations to share information between business units. Grid
technology will only be adopted if the right type of security solutions
are available.'

... snip ...

other references:
http://www-128.ibm.com/developerworks/library/gr-watch1.html
http://www.alphaworks.ibm.com/tech/optimalgrid
http://www-128.ibm.com/developerworks/grid
http://www.gridcomputingplanet.com/news/article.php/3281_1480781
http://www.ggf.org/UnderstandingGrids/ggf_grid_und
http://www.semanticgrid.org/GGF/erstand.php
http://gridcafe.web.cern.ch/gridcafe/

virtual memory

From: Anne & Lynn Wheeler <lynn@garlic.com>
Subject: Re: virtual memory
Newsgroups: comp.arch,alt.folklore.computers
Date: Fri, 02 Jun 2006 07:29:39 -0600

Brian Inglis writes:

RSX with vm? Pages (512B) were too small for such an approach: fault
context switch time and handling path length were likely longer than
user code execution time.

one of the things that some of the paging simulation work done at
the science center
http://www.garlic.com/~lynn/subtopic.html#545tech

looked at the trade-offs of different sized pages. in (severely) real
storage constrained environments ... smaller page sizes met that
applications could run efficiently with smaller real storage
requirements (i.e. portions of larger page that weren't actually
needed by an application would not need to occupy real storage).
larger page sizes tended to increase the real storage required by an
application to run ... but it tended to result in more efficient use
of the paging i/o infrastructure moving larger blocks of data to/from
memory.

this was somewhat seen in the 370 architecture from the early 70s.  it
offered both 2k and 4k page sizes. the dos/vs and vs1 operating
systems used 2k page sizes ... since they tended to run on lower end
370 machines which typically had a couple hundred k of real memory
(sometimes after fixed kernel requirements ... maybe 100k to 200k
... i.e. 50-100 2k pages ... or 25-50 4k pages). os/vs2 and vm ran
with 4k pages.

the trade-off was seen in transition of vs1 to the later 370/148 which
had 1mbyte real storage.

vs1 was adaptation of real storage MFT system to virtual memory
... somewhat akin to the earlier description of svs
http://www.garlic.com/~lynn/2006b.html#26 Multiple address spaces
http://www.garlic.com/~lynn/2006b.html#32 Multiple address spaces
http://www.garlic.com/~lynn/2006c.html#2 Multiple address spaces
http://www.garlic.com/~lynn/2006i.html#43 virtual memory

vs1 would define a single 4mbyte virtual address space from which it
ran multiple applications ... performing paging operations using 2k
pages (typically 370/135 and 370/145 with 256kbyte to 512kbyte real
storage).

in the mid-70s, the 138 and 148 follow-on to 135/145 were introduced
with 148 configuration typically having 1mbyte of real storage.

A lot of 370/148 configurations operated with vm/370 as the base host
system running vs1 in a 4mbyte virtual machine. vs1 would be
configured that its 4mbyte virtual address space would be mapped
one-for-one to the virtual machine address space. As a result, all
4mbytes of the virtual address space would appear to always be
resident and never result in a page fault. A special interface was
defined that allowed VM/370 to reflect any virtual machine page faults
to the VS1 supervisor (allowing the VS1 supervisor to perform task
switch ...  although the VS1 supervisor wouldn't otherwise have to
perform any operation in support of paging). When VM/370 finished
processing the page replacement ... it then could single the VS1
supervisor that the page was now available.

So there were some number of comparisons running VS1 on a "bare"
370/148 with 1mbyte real storage and 2k virtual pages and running VS1
on the same hardware in a virtual machine under VM/370 doing paging
with 4k virtual pages (with the total real storage available to VS1
operation reduced by the VM/370 fixed kernel requirements). It turned
out that VS1 frequently ran faster under VM/370 than it did running on
the bare hardware; i.e. VM/370 performing 4k page i/o operations (for
VS1 ... rather than relying on VS1 to perform 2k page i/o operations)
more than offset the overhead degradation of operating in a virtual
machine.

The issue was that in the mid-70s with the transition to 1mbyte (and
larger real storage sizes), systems had changed from being
significantly real-storage constrained ... to only being moderately
real-storage constrained ... and that the page fault handling and page
i/o operations were starting to dominate as source of system
bottleneck.

--
Anne & Lynn Wheeler | http://www.garlic.com/~lynn/

virtual memory

From: Anne & Lynn Wheeler <lynn@garlic.com>
Subject: Re: virtual memory
Newsgroups: comp.arch,alt.folklore.computers
Date: Fri, 02 Jun 2006 09:32:13 -0600

Del Cecchi writes:

And the 360 had something called LCS for "Large Core Storage" that was
slower and cheaper than main storage.  Of courese it wasn't used for
paging.....

I saw two different strategies in use of LCS ... a 360/65 installation
might have 1mbyte of 750nsec access memory and 8mbytes of 8mic access
LCS. one strategy was to allocate lower-use instructions and data in
LCS and access them directly in LCS. There were also some strategies
that used it sort of like an electronic disk/cache (sort of like some
of the electronic disks that were on IBM/PCs) ... where the operating
system would copy/load stuff to "fast" memory prior to useage.

--
Anne & Lynn Wheeler | http://www.garlic.com/~lynn/

virtual memory

From: Anne & Lynn Wheeler <lynn@garlic.com>
Subject: Re: virtual memory
Newsgroups: comp.arch,alt.folklore.computers
Date: Fri, 02 Jun 2006 12:15:29 -0600

Anne & Lynn Wheeler writes:

one of the things that some of the paging simulation work done at
the science center
http://www.garlic.com/~lynn/subtopic.html#545tech

re:
http://www.garlic.com/~lynn/2006l.html#9 virtual memory

part of the instruction storage ref/change trace and paging
simulation was also the program reoganization work that
was mentioned previously
http://www.garlic.com/~lynn/2006j.html#24 virtual memory

D. Hatfield & J. Gerald, Program Restructuring for Virtual Memory,
IBM Systems Journal, v10n3, 1971

which was eventually released in 1976 as vs/repack

not only were various different sized pages simulated ... but the
traces were also used for program re-organization to compact storage
(that was used together) into minimum number of pages.

the "small" page advantage in severe real storage constrained
configurations was to avoid unnecessarily bringing in parts of virtual
storage ... that weren't going to be needed ... which might be brought
in when using larger page sizes, increasing the total amount of real
memory (unnecessarily).

however with program re-organization ... there would be high
probability that adjacent storage would in-fact, be required ...  so
larger pages would tend to still occupy about the same amount of total
real storage (compared to the smaller page sizes) ... with the
advantage that the number of faults and fetches to bring in the
virtual memory was reduced (fetching chunks in fewer, larger sizes)

as real storage sizes increased ... the benefit of high compaction and
utilization from smaller pages would be less of an advantage ... and
the larger number of page faults and transfers would become an
increasingly significant throughput factor (vis-a-vis larger page
sizes with fewer faults and fetches). also, the program restructuring
tended to compact things w/o requiring the additional benefit of
smaller page sizes.

--
Anne & Lynn Wheeler | http://www.garlic.com/~lynn/

virtual memory

From: Anne & Lynn Wheeler <lynn@garlic.com>
Subject: Re: virtual memory
Newsgroups: comp.arch,alt.folklore.computers
Date: Sat, 03 Jun 2006 08:26:35 -0600

Bill Todd writes:

You don't suppose that could possibly have been because advances in
the state of the art often appear earlier in academic papers than in
commercial applications, do you?  There is, after all, a rather
considerable time-lag (often infinite, in fact) between such an
advance and its incorporation into an existing, complex OS.

for some drift ... there were academic papers and implementations of
local LRU at the time i did global LRU and it started shipping in
standard operating system ... and as referenced, it was a decade later
that there was work on global LRU in academia .... although there was
also significant (academic) pressure to suppress the publication of
the global LRU work (i've been told that there is a website/blog
someplace containing copies of all the correspondance that flew back
and forth during the disagreement).

i believe i somewhat contributed to allowing the academic global LRU
work to be published by citing real-live data showing both local LRU
AND global LRU implementations for the same operating system
running on (nearly) the same hardware with (nearly) the same kind of
workload (although the global LRU environment was running on hardware
that had only 2/3rds the real memory for paging, twice the number of
users/workload, and still had better performance than the local LRU
implementation).

a few past references:
http://www.garlic.com/~lynn/93.html#4 360/67, was Re: IBM's Project F/S ?
http://www.garlic.com/~lynn/2001c.html#10 Memory management - Page replacement
http://www.garlic.com/~lynn/2002c.html#49 Swapper was Re: History of Login Names
http://www.garlic.com/~lynn/2006i.html#37 virtual memory
http://www.garlic.com/~lynn/2006i.html#38 virtual memory
http://www.garlic.com/~lynn/2006i.html#42 virtual memory
http://www.garlic.com/~lynn/2006j.html#18 virtual memory

--
Anne & Lynn Wheeler | http://www.garlic.com/~lynn/

virtual memory

From: Anne & Lynn Wheeler <lynn@garlic.com>
Subject: Re: virtual memory
Newsgroups: comp.arch,alt.folklore.computers
Date: Sat, 03 Jun 2006 09:01:17 -0600

Bill Todd writes:

Other architectures that were not designed to cover a range of
application down into the minicomputer space.  And since VMS
included mechanisms to support multi-page paging operations, it
managed quite well in larger applications as well.

in the very early 80s, both MVS and VM shipped "big page" support for
3380s ... basically full-track transfers, ten 4k pages. there was a
number of things in big pages implementation that optimized the arm
access bottleneck *problem* that was becoming dominant system
bottleneck by that period.

lots of past posts describing/discussing "big page" implementation.
http://www.garlic.com/~lynn/2001k.html#60 Defrag in linux? - Newbie question
http://www.garlic.com/~lynn/2002b.html#20 index searching
http://www.garlic.com/~lynn/2002c.html#29 Page size (was: VAX, M68K complex instructions)
http://www.garlic.com/~lynn/2002c.html#48 Swapper was Re: History of Login Names
http://www.garlic.com/~lynn/2002e.html#8 What are some impressive page rates?
http://www.garlic.com/~lynn/2002e.html#11 What are some impressive page rates?
http://www.garlic.com/~lynn/2002f.html#20 Blade architectures
http://www.garlic.com/~lynn/2002l.html#36 Do any architectures use instruction count instead of timer
http://www.garlic.com/~lynn/2002m.html#4 Handling variable page sizes?
http://www.garlic.com/~lynn/2003b.html#69 Disk drives as commodities. Was Re: Yamhill
http://www.garlic.com/~lynn/2003d.html#21 PDP10 and RISC
http://www.garlic.com/~lynn/2003f.html#5 Alpha performance, why?
http://www.garlic.com/~lynn/2003f.html#9 Alpha performance, why?
http://www.garlic.com/~lynn/2003f.html#48 Alpha performance, why?
http://www.garlic.com/~lynn/2003g.html#12 Page Table - per OS/Process
http://www.garlic.com/~lynn/2003o.html#61 1teraflops cell processor possible?
http://www.garlic.com/~lynn/2003o.html#62 1teraflops cell processor possible?
http://www.garlic.com/~lynn/2004.html#13 Holee shit!  30 years ago!
http://www.garlic.com/~lynn/2004e.html#16 Paging query - progress
http://www.garlic.com/~lynn/2004n.html#22 Shipwrecks
http://www.garlic.com/~lynn/2004p.html#39 100% CPU is not always bad
http://www.garlic.com/~lynn/2005h.html#15 Exceptions at basic block boundaries
http://www.garlic.com/~lynn/2005j.html#51  Q ALLOC PAGE vs. CP Q ALLOC vs ESAMAP
http://www.garlic.com/~lynn/2005l.html#41 25% Pageds utilization on 3390-09?
http://www.garlic.com/~lynn/2005n.html#18 Code density and performance?
http://www.garlic.com/~lynn/2005n.html#19 Code density and performance?
http://www.garlic.com/~lynn/2005n.html#21 Code density and performance?
http://www.garlic.com/~lynn/2005n.html#22 Code density and performance?
http://www.garlic.com/~lynn/2006j.html#2 virtual memory
http://www.garlic.com/~lynn/2006j.html#3 virtual memory
http://www.garlic.com/~lynn/2006j.html#11 The Pankian Metaphor

--
Anne & Lynn Wheeler | http://www.garlic.com/~lynn/

virtual memory

From: Anne & Lynn Wheeler <lynn@garlic.com>
Subject: Re: virtual memory
Newsgroups: comp.arch,alt.folklore.computers
Date: Sat, 03 Jun 2006 10:01:30 -0600

Bill Todd writes:

That's getting repetitive, I'm afraid.  As I stated earlier, I'm
interested in head-to-head studies using repeatable traces rather than
anecdotes with far less stringent controls on several significant
variables.

re:
http://www.garlic.com/~lynn/2006l.html#12 virtual memory

when i was an undergraduate in the 60s ... i did several score
different kinds of implementations ... however, i was much more
interested in shipping the best implementation for production systems
than doing anything with academic publications. there has been very
little of the work done in that period that made in into any academic
literature. the best apples-to-apples comparison that i know of from
the period was the grenoble science center and cambridge science
center ... one doing local lru and the other doing global LRU ... both
on cp/67 operating system, both on 360/67 machines, and both with
similar cms workloads.

in the past i've periodically mentioned observing people in conflicts
spending more resources on the argument than the resources required
for each participant to go off and implement all possible variations
and measure the results.

so one possible solutation is for you to do like I did as an
undergraduate in the 60s and go off and implement several score
different variations, deploy them in production environments and
compare the measurements.

--
Anne & Lynn Wheeler | http://www.garlic.com/~lynn/

virtual memory

From: Anne & Lynn Wheeler <lynn@garlic.com>
Subject: Re: virtual memory
Newsgroups: comp.arch,alt.folklore.computers
Date: Sat, 03 Jun 2006 10:13:19 -0600

Morten Reistad writes:

ISTR 2k pages was also quite common.

previous post mentioning 370s shipping with both 2k and 4k virtual
page support and comparison of operating systems in different
environments that used 2k virtual pages with operating systems that
used 4k virtual pages ... sometimes on the same 370 hardware.
http://www.garlic.com/~lynn/2006l.html#9 virtual memory

in fact, there was an infamous customer situation that involved
upgrading from a 370/168-1 (with 16kbyte cache) to 370/168-3 (with
32kbyte cache).

the customer ran vs1 (with 2k virtual pages) under vm/370. vm/370
nominally ran with 4k virtual pages ... except doing shadow page
tables for virtual machine guest operating system .. it used the same
page mode (2k or 4k) for the shadow page tables as the virtual machine
guest operating system used for their virtual page tables.

so for the transition for 370/168-3 with twice the cache size ... they
choose to use the "2k" bit to address the additional cache line
entries (under the assumption that this was a high-end machine only
used by MVS or some vm). if the 370/168-3 ran with 2k virtual pages it
would switch to 16k cache size mode (not using the additional 16k).
if there was a switch between 4k virtual page mode and 2k virtual page
mode, it would first completely flush the cache.

so for this particular customer, the upgrade from the 370/168-1 to the
faster 370/168-3 actually resulted in degraded performance (because of
the large number of cache flushes).

--
Anne & Lynn Wheeler | http://www.garlic.com/~lynn/

virtual memory

From: Anne & Lynn Wheeler <lynn@garlic.com>
Subject: Re: virtual memory
Newsgroups: comp.arch,alt.folklore.computers
Date: Sat, 03 Jun 2006 11:10:30 -0600

Bill Todd writes:

A sufficiently large value of 'fine' to have propelled DEC to a firm
second place in the world of computing, with realistic potential to
have competed for the top spot in a finite amount of time.  The fact
that a decade or so *after* its introduction the VAX page size may
have become a bit constraining does not reflect poorly on its choice,
only on DEC's failure to have created a suitable successor in a timely
manner - and the fact that VAXen were still selling 22 years after
their introduction indicates that for at least some significant uses a
512 B page size was *still* 'fine' then.

you could also observe that cp/40 started using 4k virtual pages in
1966 ... and has gone thru several subsequent generations as cp/67,
vm/370, vm/sp, vm/xa ... and continues up thru today support 4k
virtual pages ... 40 years later.

--
Anne & Lynn Wheeler | http://www.garlic.com/~lynn/

virtual memory

From: Anne & Lynn Wheeler <lynn@garlic.com>
Subject: Re: virtual memory
Newsgroups: comp.arch,alt.folklore.computers
Date: Sat, 03 Jun 2006 11:51:50 -0600

Brian Inglis writes:

AFAIR the complaints were not generally against consolidation, but
execution of a short sighted strategy that alienated many existing
customers by making them look stupid to their management for having
bought DEC: drop upgrades and enhancements for mini and mainframe
lines, offer systems that provided no more performance than an 11/70
until the 8000 series came out five years later, support backward
compatibility with a real time OS but with interrupt latency that
prevented it from being used for much of that work, etc.

Sales seemed interested in taking orders only for VAXen, and actually
argued against purchases of 11s and 20s!

So growing customers had to look elsewhere for a forward path, unless
they had locked themselves in with extensive investments in DEC
hardware or software, and could not afford to get out.

the mid-range 370 4341 had somewhat similar issue. i've posted the us
& world-wide sales for vax, broken out by model and year
http://www.garlic.com/~lynn/2002f.html#0 Computers in Science Fiction
http://www.garlic.com/~lynn/2005f.html#37 Where should the type information be: in tags and descriptors

... and noted that 4341 sales were actually larger than vax. part
of it was some sort of threshold had been crossed and you saw large
corporations placing orders for several hundred 4341s at a time.
http://www.garlic.com/~lynn/2006k.html#31 PDP-1
http://www.garlic.com/~lynn/2006k.html#32 PDP-1

the issue for the corporation was while the 4341 offered better
price/performance than vax ... it was also better performance and
better price/performance than the entry level model of the high-end
machines.  the high-end was where the majority of the revenue was
... and even tho 4341 was larger market than vax ... it still wasn't
very significant revenue compared to the high-end.

towards the mid-80s, one of the executives of the high-end division
gave a talk that included the comment that something like 11,000 of
the vax sales should have been 4341s.
http://www.garlic.com/~lynn/2001m.html#15 departmental servers

as mentioned here the total of 750 & 780 sales (thru 1988) was
approx. 46k
http://www.garlic.com/~lynn/2002f.html#0 Computers in Science Fiction

... so the 11,000 that should have been 4341s represented nearly 1/4th
of the total combined 750 & 780 sales

however, one of the things that they failed to note was that in some
of the internal politics between the mid-range 4341 division and the
high-end division ... the high-end division managed to limit
allocation of some components needed for buidling 4341s (which was
viewed by some as trying to minimize the impact 4341 sales could have
on the high-end entry-level machines).

of course by the mid-80s, you were starting to see customers in that
mid-range market moving to workstations and high-end PCs for their
computing needs. you can also see that in the vax sale numbers in the
mid-80s ... while there was continued growth in vax sales during that
period ... much of the numbers had shifted to micro-vaxes.

as posted before ... in the later 80s the world wide corporate revenue
was approaching $60b and executives were projecting that it would
double to $120b (they turned out to be wrong, it was possibly a career
limiting move on my part at the time to start predicting that the
company was heading into the red).
http://www.garlic.com/~lynn/2006.html#21 IBM up for grabs?
http://www.garlic.com/~lynn/2006.html#22 IBM up for grabs?

this gives revenue for 86 as $51.25b and over 400k employees.
http://www-03.ibm.com/ibm/history/history/year_1986.html
and $58.6b for 1988
http://www-03.ibm.com/ibm/history/history/year_1988.html

quicky search turns up this for a little comparison, this gives DECs
revenue for the last qtr of 1985 and 1986 ($1.9b and $2.3b
respectively)
http://query.nytimes.com/gst/fullpage.html?res=9D05E0D7123AF936A25752C0A961948260

--
Anne & Lynn Wheeler | http://www.garlic.com/~lynn/

virtual memory

From: Anne & Lynn Wheeler <lynn@garlic.com>
Subject: Re: virtual memory
Newsgroups: comp.arch,alt.folklore.computers
Date: Sat, 03 Jun 2006 12:38:47 -0600

Anne & Lynn Wheeler writes:

the issue for the corporation was while the 4341 offered better
price/performance than vax ... it was also better performance and
better price/performance than the entry level model of the high-end
machines.  the high-end was where the majority of the revenue was
... and even tho 4341 was larger market than vax ... it still wasn't
very significant revenue compared to the high-end.

I've posted before about having rewritten the i/o supervisor
so the disk engineering and product test labs (bldg14 & bldg15)
could do their stuff in an operating system environment
http://www.garlic.com/~lynn/subtopic.html#disk

bldg15 had gotten the first 3033 engineering model outside of pok
... and the disk testing rarely used more than one percent of the cpu
... so we could do all sorts of stuff with the rest of the processing
power (one was to move over the air bearing simulation work that was
being used for designing the 3380 floating heads) ... recent reference:
http://www.garlic.com/~lynn/2006l.html#6 Google Architecture

similarly, got the first engineering 4341 outside of endicott ...
in fact, i had better 4341 access than the 4341 performance
test group in endicott had ... so they had me run various
of their benchmarks on the 4341 in san jose. misc. old posts
about 4341 benchmarks
http://www.garlic.com/~lynn/2000d.html#0 Is a VAX a mainframe?
http://www.garlic.com/~lynn/2000d.html#7 4341 was "Is a VAX a mainframe?"
http://www.garlic.com/~lynn/2002b.html#0 Microcode?
http://www.garlic.com/~lynn/2002i.html#7 CDC6600 - just how powerful a machine was it?
http://www.garlic.com/~lynn/2002i.html#19 CDC6600 - just how powerful a machine was it?
http://www.garlic.com/~lynn/2002k.html#4 misc. old benchmarks (4331 & 11/750)

a couple old posts comparing original 4331 & 4341 vis-a-vis vax
http://www.garlic.com/~lynn/2002k.html#1 misc. old benchmarks (4331 & 11/750)
http://www.garlic.com/~lynn/2002k.html#1 misc. old benchmarks (4331 & 11/750)
http://www.garlic.com/~lynn/2003d.html#33 Why only 24 bits on S/360?

and some approx. mip rates for later 4341 models:
http://www.garlic.com/~lynn/2002k.html#4 misc. old benchmarks (4331 & 11/750)

--
Anne & Lynn Wheeler | http://www.garlic.com/~lynn/

virtual memory

From: Anne & Lynn Wheeler <lynn@garlic.com>
Subject: Re: virtual memory
Newsgroups: comp.arch,alt.folklore.computers
Date: Sat, 03 Jun 2006 18:52:26 -0600

Peter Flass writes:

I believe I've posted before, as an observation from the trenches at
the time, that he was probably whistling in the dark. For large
numbers of users the 43xx software made it less usable than the VAX.
At the time I was managing a college computer center with mixed
administative/acaademic use of one machine, and the VAX was definitely
a better fit than the 4341.  I loved the 4341, but neither MVS nor VM
would have really worked in our environment.

For the large corporations, no doubt compatibility with existing
mainframe systems swung the balance the other way.

re:
http://www.garlic.com/~lynn/2006l.html#18 virtual memory

there was a share study that observed that 4341 had a lot more
infrastructure support requirement than vax ... so while 4341 systems
may have been better system price/performance ... there was
requirement for scarce resources to support the machine that were hard
to find (especially in smaller shops looking at only a machine or two,
it was a lot easier to amortize such resources over several hundred or
a thousand or two such machines).

we had done quite a bit of work somewhat in conjunction with the ecps
effort for the 148 ... random reference:
http://www.garlic.com/~lynn/94.html#21 370 ECPS VM microcode assist

(follow-on to the 145 and precursor to the 4341) that signifantly
addressed lots of those issues.  however, that part of the effort was
rejected by corporate ... it wasn't clear whether they were ambivalent
about moving various aspects of mainframe into more ease of use
direction (also part of it included effectively incorporating virtual
machine support into base of all shipped machines ... somewhat
analogous to modern day lpar support ... which corporate wasn't ready
for either ... they still thot there was some chance of killing off
virtual machine operation).

--
Anne & Lynn Wheeler | http://www.garlic.com/~lynn/

Why I use a Mac, anno 2006

From: Anne & Lynn Wheeler <lynn@garlic.com>
Subject: Re: Why I use a Mac, anno 2006
Newsgroups: comp.sys.mac.advocacy,alt.folklore.computers
Date: Sun, 04 Jun 2006 11:13:37 -0600

George Graves writes:

Are you trying to tell us that Windows IS secure? That dog won't hunt,
my friend.

recent thread mentioning the eu finread standard ... work on the
standard was in part because of widely held view that most PCs have a
large number of vulnerabilities
http://www.garlic.com/~lynn/aadsm23.htm#45 Status of SRP
http://www.garlic.com/~lynn/aadsm23.htm#49 Status of SRP
http://www.garlic.com/~lynn/aadsm23.htm#50 Status of SRP

--
Anne & Lynn Wheeler | http://www.garlic.com/~lynn/

Virtual Virtualizers

From: Anne & Lynn Wheeler <lynn@garlic.com>
Subject: Re: Virtual Virtualizers
Newsgroups: comp.arch
Date: Sun, 04 Jun 2006 11:07:50 -0600

jsavard@excxn.aNOSPAMb.cdn.invalid (John Savard) writes:

Clearly, if it is called upon to simulate operations that it didn't
expect to have to simulate, the basic idea of virtualization would be
broken - the program would learn it didn't have privileges in reality
that it thought it had.

Thus, it seemed to me that the hardware would have to keep track of
*three* copies of the program status word. One shows the privilege level
that a virtualized program thinks it has, a second shows the privilege
level it actually has - and a third shows the privilege level that its
parent *thinks* it actually has.

When an operation is attempted by a child process which the parent
process believed it could delegate, then one has to skip levels
*upwards* to find the process that would actually do the simulation.

Is this an issue anyone has actually dealt with in real architectures,
or has virtualization heretofore only allowed machines without the
ability to have virtual machine child processes to be virtualized?

virtualization could be recursive to arbitrary depth.

i've posted several times before joint project in early 70s (35 years
ago) between cambridge
http://www.garlic.com/~lynn/subtopic.html#545tech

and endicott to provide 370 virtual machines under cp67 ... in part
because there were misc. instruction and field definitions between
360/67 virtual memory structure and 370 virtual memory structure;
probably the first use of the internal network in support of a
distributed project
http://www.garlic.com/~lynn/subnetwork.html#internalnet

first, was to create a version of cp67 kernel that provided 370
virtual machines ... called "cp67h". then there was a modification of
cp67 to run in 370 virtual machine (rather than 360 called "cp67i").

virtual memory hadn't yet been announced for 370 so it was suppose to
be a super secret. the cambridge cp67 provided access to
non-employees, included students from various educational institutions
in the cambridge area (mit, harvard, bu, etc) ... so there was concern
if cp67h (providing 370 virtual machines) were to run on the bare
hardware ... that details of 370 virtual memory would leak.

so cp67h was normally run in 360/67 virtual machine under cp67l (which
was running on the real 360/67 hardware)

so standard operation was something like

cms running in a 370 virtual machine provided by cp67i
 cp67i running in a 370 virtual machine provided by cp67h
  cp67h running in a 360/67 virtual machine provided by cp67l
   cp67l running on real 360/67 machine

so there was recursive tracking of the PSW ... however, the managing
PSW status was rather trivial compared to managing the shadow page
tables.

you can sort of think of shadow page tables as the TLB for the virtual
machine.

cp67i had a set of virtual page tables that mapped cms virtual address
space addresses to the addresses into cp67i's (virtual) machine
addresses.

cp67h had to simulate cp67i's page tables with shadow page tables.
there were replication of cp67's page tables but having cp67h
(virtual) machine addresses substituted for the addresses in cp67i's
page tables.

then cp67l had to simulate cp67h's page tables (including cp67h's
shadow tables simulating cp67i's page tables) using shadow tables that
were duplicates of cp67h's page tables ... but substituting the "real"
machine addresses for the cp67h (virtual) machine addresses.

misc. past posts mentioning cp67l, cp67h, cp67i stuff
http://www.garlic.com/~lynn/2002j.html#0 HONE was .. Hercules and System/390 - do we need it?
http://www.garlic.com/~lynn/2004b.html#31 determining memory size
http://www.garlic.com/~lynn/2004h.html#27 Vintage computers are better than modern crap !
http://www.garlic.com/~lynn/2004p.html#50 IBM 3614 and 3624 ATM's
http://www.garlic.com/~lynn/2005c.html#59 intel's Vanderpool and virtualization in general
http://www.garlic.com/~lynn/2005d.html#66 Virtual Machine Hardware
http://www.garlic.com/~lynn/2005g.html#17 DOS/360: Forty years
http://www.garlic.com/~lynn/2005h.html#18 Exceptions at basic block boundaries
http://www.garlic.com/~lynn/2005i.html#39 Behavior in undefined areas?
http://www.garlic.com/~lynn/2005j.html#50 virtual 360/67 support in cp67
http://www.garlic.com/~lynn/2005p.html#27 What ever happened to Tandem and NonStop OS ?
http://www.garlic.com/~lynn/2006.html#38 Is VIO mandatory?
http://www.garlic.com/~lynn/2006e.html#7 About TLB in lower-level caches
http://www.garlic.com/~lynn/2006f.html#5 3380-3390 Conversion - DISAPPOINTMENT

misc. past posts mentioning shadow page tables:
http://www.garlic.com/~lynn/94.html#48 Rethinking Virtual Memory
http://www.garlic.com/~lynn/2002l.html#51 Handling variable page sizes?
http://www.garlic.com/~lynn/2003b.html#0 Disk drives as commodities. Was Re: Yamhill
http://www.garlic.com/~lynn/2003d.html#69 unix
http://www.garlic.com/~lynn/2003g.html#18 Multiple layers of virtual address translation
http://www.garlic.com/~lynn/2004c.html#35 Computer-oriented license plates
http://www.garlic.com/~lynn/2004l.html#67 Lock-free algorithms
http://www.garlic.com/~lynn/2004o.html#18 Integer types for 128-bit addressing
http://www.garlic.com/~lynn/2005d.html#58 Virtual Machine Hardware
http://www.garlic.com/~lynn/2005d.html#66 Virtual Machine Hardware
http://www.garlic.com/~lynn/2005d.html#70 Virtual Machine Hardware
http://www.garlic.com/~lynn/2005h.html#11 Exceptions at basic block boundaries
http://www.garlic.com/~lynn/2005h.html#17 Exceptions at basic block boundaries
http://www.garlic.com/~lynn/2005h.html#18 Exceptions at basic block boundaries
http://www.garlic.com/~lynn/2005j.html#38 virtual 360/67 support in cp67
http://www.garlic.com/~lynn/2005k.html#39 Determining processor status without IPIs
http://www.garlic.com/~lynn/2005k.html#42 wheeler scheduler and hpo
http://www.garlic.com/~lynn/2005n.html#47 Anyone know whether VM/370 EDGAR is still available anywhere?
http://www.garlic.com/~lynn/2005o.html#8 Non Power of 2 Cache Sizes
http://www.garlic.com/~lynn/2005p.html#45 HASP/ASP JES/JES2/JES3
http://www.garlic.com/~lynn/2006e.html#0 About TLB in lower-level caches
http://www.garlic.com/~lynn/2006e.html#6 About TLB in lower-level caches
http://www.garlic.com/~lynn/2006e.html#7 About TLB in lower-level caches
http://www.garlic.com/~lynn/2006e.html#12 About TLB in lower-level caches
http://www.garlic.com/~lynn/2006e.html#19 About TLB in lower-level caches
http://www.garlic.com/~lynn/2006e.html#20 About TLB in lower-level caches
http://www.garlic.com/~lynn/2006e.html#37 The Pankian Metaphor
http://www.garlic.com/~lynn/2006i.html#31 virtual memory
http://www.garlic.com/~lynn/2006l.html#15 virtual memory

--
Anne & Lynn Wheeler | http://www.garlic.com/~lynn/

Virtual Virtualizers

From: Anne & Lynn Wheeler <lynn@garlic.com>
Subject: Re: Virtual Virtualizers
Newsgroups: comp.arch,alt.folklore.computers
Date: Sun, 04 Jun 2006 11:59:29 -0600

jsavard@excxn.aNOSPAMb.cdn.invalid (John Savard) writes:

On my web page, at

http://www.quadibloc.com/arch/ar0703.htm

I note a subtle precaution that needs to be taken on a computer
architecture which allows full, unrestricted virtualization - including
the ability to run virtualization software *in a virtual machine*.

Basically, the snag I saw was this:

A program running in a virtual machine may believe itself to have
privileges it does not; when it executes an instruction covered by this,
its parent simulates them for it.

What happens, though, when that program is itself a parent - and it
includes, among the native privileges given to a child process, some
which it only *believes* itself to posess?

re:
http://www.garlic.com/~lynn/2006l.html#21 Virtual Virtualizers

so the original VMA (virtual machine assist) was done on the
370/158 (and then replicated on the 370/168.

vm370 would load a special value in control register six (CR6) that
was otherwise defined as unused/reserved in 370 architecture.

normally virtual machines were run in "problem" state ... and all
"privileged/supervisor" state instructions would interrupt into the
vm370 hypervisor for simulation. with CR6 loaded, there was support
for some supervisor state instructions to be executed in problem state
using virtual machine rules.

vm370 ran into some issues with this with cms shared, protected pages.
for the morph from cp67/cms to vm370/cms, cms was reorganized to take
advantage of the vm370 architecture for 64k (16 4k page) segment
"protect". segment table entry for different virtual address spaces
could point at the same page table (allowing multiple different
virtual address spaces to share the same real pages). 370 virtual
memory architecture provided for a "read-only" bit in segment table
entry.

however, the hardware development to retrofit virtual memory to
370/165 got something like six months behind schedule. in an
escalation meeting, the 165 engineers said they could make up six
months if certain features from 370 virtual memory architecutre was
dropped ... one of those features was segment protection. that
proposal carried the day ... however, the cms shared segment (having
multiple different virtual address spaces sharing the same real pages
in a protect manner) was impacted.

so, in order to preserve the capability of sharing common real pages
... it was necessary to drop back to a hack using standard storage
protection (carried forward from 360 architecture). this provided
setting a storage protect key associated with every 2k block of
storage (which could apply just store protect or both store & fetch
protect). the program status word then carried a corresponding storage
protect key. a PSW storage protect key of zero allowed instructions to
access/modify any storage. A non-zero PSW storage protect key met that
instructions could only access/modify 2k blocks of storage that had
storage protect key that matched the PSW storage protect key.

So CMS virtual address spaces were run with a hack that forced all
zero storage protect keys (in the cms virtual address space) to
non-zero. Furthermore, any new (virtual) PSW loaded by CMS with a zero
(PSW) storage protect key was forced to non-zero. The problem was that
the virtual machine microcode assist (VMA) only had the rules for
standard virtual machine privilege instruction operation (it didn't
provide for supporting the fiddling of storage protect keys). As a
result, CMS virtual address spaces with "shared segments" had to be
run with VMA disabled. It was possible to run a CMS virtual address
space w/o any shared segments ... and have VMA enabled ... but the
advantage of running with shared real pages (and w/o VMA) was normally
much larger than the advantage of running with VMA enabled (and w/o
shared real pages).

VMA was signficiantly expanded for the 370/xa on 3081 with
SIE instruction. recent post referencing an old SIE discussion
http://www.garlic.com/~lynn/2006j.html#27 virtual memory

amdahl then introduced "hypervisor" which was a virtual machine subset
all implemented in hardware/microcode (not requiring a software
virtual machine hypervisor) ... also mentioned in the SIE ref.

response was "PR/SM" on the 3090 ... which offered similar capability.
this evolved into LPARs. Initially PR/SM provided assist for only one
level "deep" ... i.e. if a software virtual machine operating system
was running in a PR/SM (LPAR) hypervisor ... a lot of the virtual
machine assist for its virtual machine was not operational.

there have been recent discussions in various mainframe n.g. about
PR/SM being upgraded to handle two level "deep" operation ... i.e.
virtual machines running under a software virtual machine operating
system which, in turn, was running in PR/SM (LPAR) hypervisor, would
have virtual machine microcode assists operational.

misc. past posts about the cms storage protect hack for shared
segments ... after shared segment protect was dropped from 370 virtual
memory architecture:
http://www.garlic.com/~lynn/2005.html#6 [Lit.] Buffer overruns
http://www.garlic.com/~lynn/2005c.html#18 [Lit.] Buffer overruns
http://www.garlic.com/~lynn/2005f.html#45 Moving assembler programs above the line
http://www.garlic.com/~lynn/2005f.html#46 Moving assembler programs above the line
http://www.garlic.com/~lynn/2005h.html#9 Exceptions at basic block boundaries
http://www.garlic.com/~lynn/2005h.html#10 Exceptions at basic block boundaries
http://www.garlic.com/~lynn/2006.html#13 VM maclib reference
http://www.garlic.com/~lynn/2006i.html#9 Hadware Support for Protection Bits: what does it really mean?
http://www.garlic.com/~lynn/2006i.html#24 Virtual memory implementation in S/370
http://www.garlic.com/~lynn/2006j.html#5 virtual memory
http://www.garlic.com/~lynn/2006j.html#41 virtual memory

--
Anne & Lynn Wheeler | http://www.garlic.com/~lynn/

Virtual Virtualizers

From: Anne & Lynn Wheeler <lynn@garlic.com>
Subject: Re: Virtual Virtualizers
Newsgroups: comp.arch
Date: Sun, 04 Jun 2006 12:17:35 -0600

Anne & Lynn Wheeler writes:

so there was recursive tracking of the PSW ... however, the managing
PSW status was rather trivial compared to managing the shadow page
tables.

re:
http://www.garlic.com/~lynn/2006l.html#21 Virtual Virtualizers

the other "shadow" process is for virtual machine I/O programs.  the
operating system in the virtual machine is building I/O programs using
what it thinks are "real addresses" (but in fact are virtual addresses
of the virtual machine).

for i/o programs, the supervisor implementating the virtual machine
support has to make a "shadow" copy of the i/o program in the virtual
machine and then swizzle all the virtual machine addresses to "real"
addresses (as well a pinning the associated virtual pages to the real
addresses for the duration of the i/o).

with regard to sie, pr/sm and recursive virtual machine support by
hardware virtual machine performance assists ...
http://www.garlic.com/~lynn/2006l.html#22 Virtual Virtualizers

here is a discussion of some of the changes for zSeries hardware
http://www.vm.ibm.com/perf/tips/z890.html

--
Anne & Lynn Wheeler | http://www.garlic.com/~lynn/

Google Architecture

From: Anne & Lynn Wheeler <lynn@garlic.com>
Subject: Re: Google Architecture
Newsgroups: bit.listserv.ibm-main,alt.folklore.computers
Date: Mon, 05 Jun 2006 07:36:00 -0600

Craddock, Chris wrote:

Pat Helland (formerly with Tandem and MS, now with Amazon) has written
some very lucid and entertaining discussions about how economics are
changing their system design points. He was one of the originators of
the Tandem Non-Stop transaction system and a life-long transaction
processing bigot. Now he's talking openly about his ACID apostasy. If
Pat is ready to cast that aside, I think everyone else ought to at least
take it seriously.

re:
http://www.garlic.com/~lynn/2006l.html#6 Google Architecture

a lot of the ACID (and TPC) stuff originated with Jim. When Jim left
system/r group (original relational/sql implementation):
http://www.garlic.com/~lynn/submain.html#systemr

and went to tandem, we would frequently drop by and visit him. In fact,
I got blamed for something called tandem memos ... couple posts with
minor refs:
http://www.garlic.com/~lynn/2005c.html#50
http://www.garlic.com/~lynn/2006h.html#9

Later when we were doing ha/cmp (on non-mainframe platform)
http://www.garlic.com/~lynn/subtopic.html#hacmp

and out preaching availability and scale-up on commodity priced hardware
http://www.garlic.com/~lynn/95.html#13

Jim and I had some disagreements ... of course he was with DEC at the
time, and they were pitching vax/clusters.

Of course, later he was up on the stage announcing commodity priced
clusters for scale-up and availability.

for other drift ... a recent thread discussing some vax/vms and
mid-range mainframe market somewhat from late 70s into mid and
late 80s:
http://www.garlic.com/~lynn/2006l.html#17
http://www.garlic.com/~lynn/2006l.html#18
http://www.garlic.com/~lynn/2006l.html#19

Mainframe Linux Mythbusting (Was: Using Java in batch on z/OS?)

From: Anne & Lynn Wheeler <lynn@garlic.com>
Subject: Re: Mainframe Linux Mythbusting (Was: Using Java in batch on z/OS?)
Newsgroups: bit.listserv.ibm-main,alt.folklore.computers
Date: Mon, 05 Jun 2006 10:29:58 -0600

Ed Gould wrote:

Timothy:

I profess I have never installed VM (unless you count once 30 years
ago). That being said, its never the d/l'ing that is the difficult part
.. its always the "post" downloading that gets to be a PITA and always
requires some (some say quite a bit) expertise. The VM system I helped
set up was on a 4331 (w/3310's IIRC) was not a breeze by any stretch.
IIRC it was a IPO (but I could be remembering incorrectly). But to get
back to MVS the same thing can be said with a SERVPAC. I won't talk too
much about servpac's as I have already indicated my dislike for them on
here in the past. The d/l is almost never hard the customization is
always the gotcha IMO. So please don't say the install is easy as it is
only about a 1/3 (1/4?) of the job. You are making a broad statement,
IMO, and putting a broad brush on the effort and thereby making it seem
like any idiot can do so. This seems to be an effort by IBM (starting in
the 1990's) that sysprogs are no longer needed. IBM (even in the SERVPAC
classes) discussed that they are no longer needed that any joe blow can
do a servpac.

I am not picking on LE but if you take the defaults that LE put out, a
lot of your batch programs will not work correctly. The same can be said
for other "optional" customization items that need careful monitoring at
customization times. Most likely an untrained sysprog would take all the
defaults. I was training a sysprog at the time of the servpac and I gave
him the chore to determine which customization jobs were needed and he
didn't have a clue. Like I said I don't wish to pick on ONE component
but LE is a good target (sigh).

IBM (and you) seems to be sending out signals that we sysprogs are no
longer needed. I am lucky to be in retirement and not have to put up
with this BS from IBM anymore.

we did a lot of work for vm originally on 138/148 .... besides ecps
http://www.garlic.com/~lynn/94.html#21 370 ECPS VM microcode assist

there was a lot of investigation trying to make it almost as
transparently part of the machine as current day LPAR. ... basically
pre-installed with lots of useability stuff on every machine that went
out the door

however this was back in the days when corporate still thot they had a
chance to kill vm ... and while they allowed endicott to ship ecps
support, the idea that every machine that went out the door had it
pre-installed was blocked (along with the lots of the usability stuff)

POK had the vm development group in burlington mall shutdown and all the
people were told they had to move to POK to work on the (internal only)
VMTOOL supporting mvs/xa development (justification was that mvs/xa
development couldn't meet schedule unless they had all the vm developers
working on it also) ... and there would be no more vm products for
customers. endicott managed to pickup some of the vm370 mission and
rescue some of the people from having to move to POK (although quite a
few stayed in the boston area and went to places like DEC to work on
what was to became VMS).

however, this (138/148) was the leading edge of some companies starting
to order the boxes in large numbers. this really accelerated in the
4331/4341 time-frame where it wasn't unusual to have customers ordering
the boxes in a couple hundred at a time. old reference to one such situation
http://www.garlic.com/~lynn/2001m.html#15 departmental servers

the issue started becoming if the number of machines increase by two
orders of magnitude (100 times) ... where does the two orders of
magnitude increase in the number of support people come from?

this period also saw a big explosion in the number of vm/4341s on the
internal network
http://www.garlic.com/~lynn/subnetwork.html#internalnet

which was nearly almost all vm machines already. at the time the arpanet
cutover to internetworking protocol on 1/1/83, there was possibly 100
arpanet nodes with somewhere between 100 and 255 connected system hosts
http://www.garlic.com/~lynn/subnetwork.html#internet

however the internal network was almost 1000 nodes by that time ...
almost all vm (and non-sna) machines. a recent thread
http://www.garlic.com/~lynn/2006k.html#40 Arpa address
http://www.garlic.com/~lynn/2006k.html#42 Arpa address
http://www.garlic.com/~lynn/2006k.html#43 Arpa address

vax was also selling into that same mid-range market (as 4331 and 4341).
there was some study that 4341 was better price/performance and a claim
that something like 11,000 vax sales should have been 4341s ... recent
post mentioning the subject:
http://www.garlic.com/~lynn/2006l.html#17 virtual memory

however, in that period there was a SHARE study/report that found that a
lot of customers were buying vax (instead of vm/4341s) because of much
less resources/skills were needed to install, support and maintain the
systems (although overall 4331/4341 did still sell more total than vax,
in part because of large customers ordering them a couple hundred at a
time).

then there was an expectation of a similar explosion in sales for the
4381 (4341 follow-on) ... but by that time customers were starting to
buy workstations and high-end PCs for that market segment. you can also
see similar affect on the vax sales going into the mid-80s.
http://www.garlic.com/~lynn/2006k.html#31
http://www.garlic.com/~lynn/2002f.html#0

The issue in the 80s (especially early 80s) wasn't that sysprogs were no
longer (absolutely) needed ... it was if you had a customer with a
couple large mainframes in a datacenter ... and they ordered 210 4341s
http://www.garlic.com/~lynn/2001m.html#15 departmental servers

where was the additional trained staff going to come from?

Google Architecture

From: Anne & Lynn Wheeler <lynn@garlic.com>
Subject: Re: Google Architecture
Newsgroups: bit.listserv.ibm-main,alt.folklore.computers
Date: Tue, 06 Jun 2006 06:55:05 -0600

Phil Payne wrote:

And Google's database(s) are also hopelessly out of date.  I have logs
from my web site that show Google (despite its "sitemaps" programme)
simply hasn't spidered changed pages for weeks.  MSN, Yahoo, Ask (and
even IBM Almaden) visit much more frequently.

i have the opposite experience ... the major guys (including google)
each appear to hit every html file several times a day. almaden is maybe
1/10th that ... but as i've mentioned before, i suspect they are all
using it as a test/regression case (possibly because of the very large
ratio of hrefs to text).

much of the information is maintained with information base technology
(that i developed) ... and applications are used to generate/update the
html files. for instance, the merged financial taxonomy and glossary has
greater than 35,000 hrefs in a single file.
http://www.garlic.com/~lynn/index.html#glosnote
http://www.garlic.com/~lynn/financial.htm

the ietf rfc index files currently have an aggregate of nearly 125,000
hrefs.
http://www.garlic.com/~lynn/rfcietff.htm

a few recent posts mentioning the subject:
http://www.garlic.com/~lynn/2006h.html#35 64-bit architectures & 32-bit instructions
http://www.garlic.com/~lynn/2006i.html#11 Google is full
http://www.garlic.com/~lynn/2006l.html#0 history of computing

Google Architecture

From: Anne & Lynn Wheeler <lynn@garlic.com>
Subject: Re: Google Architecture
Newsgroups: bit.listserv.ibm-main,alt.folklore.computers
Date: Tue, 06 Jun 2006 07:08:27 -0600

R.S. wrote:

The only thing I can complement is there is not better engine than
Google, is there ?
So, I'm going to keep using google, until find something better. Can be
mainframe based if you want. Or audi (car) based, I don't care.

BTW: outdated pages are quite useful somtimes. I found the information
which was already deleted from original page. I can use oudated copy.
I like it. Usually such pages are far from first three hits.

re:
http://www.garlic.com/~lynn/2006l.html#4 Google Architecture
http://www.garlic.com/~lynn/2006l.html#6 Google Architecture
http://www.garlic.com/~lynn/2006l.html#7 Google Architecture
http://www.garlic.com/~lynn/2006l.html#8 Google Architecture
http://www.garlic.com/~lynn/2006l.html#24 Google Architecture
http://www.garlic.com/~lynn/2006l.html#26 Google Architecture

then there is the way-back machine. in a thread looking at UK
chip&pin vulnerabilities, a URL (from a few years ago) that went
into some discussion of the yes card vulnerability, appeared to have
become cloaked. however it was still available at the way-back
machine.

parts of fraud & vulnerability thread mentioning yes cards ... and
mentioning old web page still at the way-back machine
http://www.garlic.com/~lynn/aadsm22.htm#20 FraudWatch - Chip&Pin, a new tenner (USD10)
http://www.garlic.com/~lynn/aadsm22.htm#23 FraudWatch - Chip&Pin, a new tenner (USD10)
http://www.garlic.com/~lynn/aadsm22.htm#29 Meccano Trojans coming to a desktop near you
http://www.garlic.com/~lynn/aadsm22.htm#33 Meccano Trojans coming to a desktop near you
http://www.garlic.com/~lynn/aadsm22.htm#34 FraudWatch - Chip&Pin, a new tenner (USD10)
http://www.garlic.com/~lynn/aadsm22.htm#39 FraudWatch - Chip&Pin, a new tenner (USD10)
http://www.garlic.com/~lynn/aadsm22.htm#40 FraudWatch - Chip&Pin, a new tenner (USD10)
http://www.garlic.com/~lynn/aadsm22.htm#47 Court rules email addresses are not signatures, and signs death warrant for Digital Signatures

Google Architecture

From: Anne & Lynn Wheeler <lynn@garlic.com>
Subject: Re: Google Architecture
Newsgroups: bit.listserv.ibm-main,alt.folklore.computers
Date: Tue, 06 Jun 2006 08:05:11 -0600

Phil Payne wrote:

Google's solution is simply not scaling.  Period.  Check out the
complaints of massive page loss both on 28th March and 26th April.

A lot of people have been suggesting that Google might move to
"mainframes" - although they don't seem to mean zSeries.  Perhaps a
POWER or BladeServer solution.  Perhaps Superdome.  Someone from IBM
should be talking to these people.  And probably is.

As for the currency of Google's results - they're ordure (avoiding too
many netnanny bounces).  I can prove it - I have the logs and I've
posted details in many of the webmaster forums.

The most intensive spiderer (?) at present is Yahoo, by a country
mile.  The most up to date index is MSN - no doubt whatsoever about
it.  Ask is pretty well up there too.  Google is MILES behind on both
currency and content.

Search Usenet for "the Google dance".  Check out
http://google.fergusons.dk every now and then.

(P.S.  Object REXX is just GREAT for web server log analysis.  Perhaps
SAS would be better, but $$$$.)

there is possibly two different subthreads here ... the massively
parallel, non-mainframe technology in use by all the major search
engines are unlikely ever to migrate to mainframes ... and the
quantity/quality of the google massively parallel, non-mainframe
implementation vis-a-vis the quantity/quality of the other massively
parallel, non-mainframe implementations.

i'm seeing nearly identical total daily hits from google and yahoo
crawlers ... however, the yahoo hits are coming from a much larger
number of different, unique originating ip-addresses

Mainframe Linux Mythbusting (Was: Using Java in batch

From: Anne & Lynn Wheeler <lynn@garlic.com>
Subject: Re: Mainframe Linux Mythbusting (Was: Using Java in batch
 on z/OS?)
Newsgroups: bit.listserv.ibm-main,alt.folklore.computers
Date: Tue, 06 Jun 2006 11:57:52 -0600

Marian Gasparovic wrote:

Why are mainframe people so reluctant to change ? I know cases where
mainframe people refused to implement new applications, so they were
implemented on different platform, old applications were removed as
well as mainframe. I witnessed this situation personaly at one
customer before I joined IBM. Now when I work for IBM in mainframe
market I know the fights we have to fight. Mainframe platform and
people are perceived as least flexible. I repeat - we are perceived as
least flexible.

re:
http://www.garlic.com/~lynn/2006l.html#25 Mainframe Linux Mythbusting (Was: Using Java in batch on z/OS?)

very Boyd and OODA-loop oriented
http://www.garlic.com/~lynn/subboyd.html#boyd
http://www.garlic.com/~lynn/subboyd.html#boyd2

I sponsored Boyd a number of times at internal corporate seminars in
the early 80s (it started out being slightly less than full day, but
as Boyd evolved some of his talks it began harder and harder to get it
all crammed into a single day) .... some number of people would go in
thinking it would be a very non-civilian oriented talk and were
surprised to find how applicable it was to all sorts of business
endeavors (it has since become much more acceptable to reference
boyd's OODA-loop concepts in business settings, especially where rapid
adaptation and agility to deal with changing circumstances is
required).

having projected in the late 80s that things were headed into the red
and things would have to change ... in the early 90s, we would
periodically visit places like somers for discussions on the subject.
nobody would really disagree that things were going to have to change
... but you go back a couple months later and found nothing was
changing.

one possible observation was that there were a large number of senior
people with possibly 25-30 years experience and their perceived value
was based on experience in on a long standing, relatively consistent
paradigm. any significant changes in the paradigm would significantly
reduce the perceived value of these individuals' experience.

there appeared to be a large contingent of people that didn't disagree
that things were going to have to change ... but they were doing
everything possible to forestall it until at least after they,
personally had retired (and then it would be somebody else's problem)

One or two CPUs - the pros & cons

From: Anne & Lynn Wheeler <lynn@garlic.com>
Subject: Re: One or two CPUs - the pros & cons
Newsgroups: bit.listserv.ibm-main,alt.folklore.computers
Date: Tue, 06 Jun 2006 15:08:00 -0600

Charles Mills wrote:

Are you sure? That's totally contrary to my impression.

There are three states for the above machine:

- both tasks waiting for I/O
- one task waiting for I/O and the other task computing
- either both tasks computing, or if a single CPU, one computing and the
other ready-to-run and waiting for the CPU

Clearly processor speed is irrelevant to the first state. For the second
state, a single, faster processor is clearly an advantage, because the
single running task will run faster (and could not take any advantage of two
CPUs). For the final state, you either have one task running at "200 MIPS"
or two tasks running at "100 MIPS" - roughly equivalent situations from a
thruput point of view. So clearly, the two 100-MIPS CPUs are no faster in
the first state, slower in the second state, and no faster in the third
state - and therefore almost certainly slower, not faster, overall. (Even
before we consider the multi-processor overhead that you alluded to in your
full post.)

for two processor SMP ... an SMP kernel can add possibly 20-30percent
overhead (your mileage may vary) compared to uniprocessor kernel running
on a single processor machine.

370s had extremely strong memory consistency and for cache operations
... a two-processor 370 SMP would run the processor hardware at 90
percent of a uniprocessor ... to allow for handling cross-cache
consistency chatter ... so the bare two-processor hardware started out
at 1.8times that of a uniprocessor. you added in typical smp kernel
overhead and a two-processor smp got something like 1.5 times the
thruput of a uniprocessor.

there were games I played with highly optimized and extremely
efficient smp kernel processing along with some games related to cache
affinity ... and sometimes you could come out with thruput greater
than two times a uniprocessor (having twice the cache size and some
games with cache affinity and cache hit ratios more than compensating
for the base two-processor machine hardware running only 1.8times a
single processor).

when 3081 came out it was only going to be in multiprocessor versions
(so the uniprocessor vis-a-vis multiprocessor slow-down wasn't going
to be evident). however ACP/TPF didn't have multiprocessor support and
that represented a significant customer base. Frequently you found
ACP/TPF running under VM on 3081 (solely using VM to managed two
processor operation). eventually they were forced into coming out with
the single processor 3083 ... which had individual processor that ran
at nearly 15 percent faster than 3081 processor (because of the
elimination of the slow-down provisions for cross-cache chatter)

running the processors (in multiprocessor mode) at only .9 (that of
uniprocessor to allow for cross-cache chatter) was only the start. any
actual cross-cache chatter could result in even further hardware
thruput degradation. going to the four-processor 3084 ... the amount
of cross-cache chatter effects got worse (in the two-processor case, a
cache was getting signals from one other cache, in the four-processor
case, a cache was getting hit with signals from three other caches).

in that time-frame you saw both the VM and MVS kernels restructured so
that internal kernel structures and storage management was carefully
laid out on cache-line boundaries and done in multiples of cache-lines
... to reduce the impact of stuff like cache-line thrashing. That
restructuring supposedly got something like a five percent overall
system thruput increase.

there was some joke that to compensate for the smp cache effects, the
3090 caches used a machine cycle ten times faster than that of the
3090 processor machine cycle.

there can be secondary considerations. in the 158/168 time-frame ...
370/158-3 at around 1mip processing was about at the knee of the
price/technology curve. the 370/168-3 at around 3mip processing was
way past the knee of price/technology curve ... and cost significantly
more to build/manufacture.

at one point we had a project called logical machines to build a
16-way smp using 158-3 engines ... that still cost less to manufacture
(parts plus build) than a single processor 168. we were going great
guns ...  until some upper executive realized that MVS was never going
to be able to ship 16-way SMP support within the life-time of the
project and killed the effort (in part because it wouldn't look good
if there was a flagship 16-way smp hardware product and there was no
MVS capable of running on it). we had also relaxed some cache
consistency requirements ... that made it much less painful getting to
16-way smp operation.

something similar to your description of machine processing states has
also been used in the past to describe the overhead of virtual machine
operation. if the guest is in wait state ... the amount of virtual
machine overhead is zero. if the guest is executing only problem state
instructions, the virtual machine overhead is zero. it isn't until you
start executing various supervisor state instructions that you start
to see various kinds of virtual machine overhead degradation.

Google Architecture

From: Anne & Lynn Wheeler <lynn@garlic.com>
Subject: Re: Google Architecture
Newsgroups: bit.listserv.ibm-main,alt.folklore.computers
Date: Wed, 07 Jun 2006 06:19:23 -0600

Phil Smith III wrote:

No, Google succeeds because "Good enough is good enough" (SM,
me).  It works well enough to satisfy end-users, so they use it.  Yes,
Big Daddy is a problem; yes, many webmasters are unhappy; but Google
continues to work well enough to power the Internet economy
(high-falutin' words, but, in my experience, NOT overblown).
The complaints about varied results, stale pages, intermittent
spidering, etc. go back to ... Well, forever.  They're evidence of
suboptimal-ness, not "failure" or "a sham".  I'm not in love
with Google, have no stake in them (wish I did!), but the vitriol
heaped upon them is unreasonable.  Google works, period.  10**n
successful searches per day prove that empirically.

about ten years ago, i had opportunity to spend some time with people at
NIH's national library of medicine. at the time, they had a mainframe
bdam implementation that dated from the late 60s. two of the people that
had worked on the original implementation from the 60s were still
around. we were able to exchange some war stories ... because i had an
opportunity to be at university that was involved in the original cics
product beta test. they had an onr grant to do a library project and i
got to shoot cics and bdam bugs.

at the time, somebody commented that there was something like 40k
professional nlm librarians world-wide. the process was that they would
sit down with a doctor or other medical professional for a couple hrs,
take down their requirements and then go off for 2-3 days and do
searches ... eventually coming back with some set of results.

nlm had passed the search threashold of extremely large number of
articles back around 1980 and had a severe bimodel keyword search
problem. out to five to eight keywords ... there would still be hundreds
of thousands of responses ... and then adding one more keyword (to the
search) would result in zero responses. the holy grail of large search
infrastructures has been to come up with the number of responses greater
than zero and less than a hundred.

in early 80s, nlm got a interface, grateful med. grateful med could ask
for the return of the number of responses ... rather than the actual
responses. grateful med would keep tracks of searches and count of
responses. the person doing the search seemed to involve a slightly,
semi-directed random walk ... looking for a query that met the holy
grail ... greater than zero responses and less than one hundred.

finding acceptable responses is problem common to most environments
after the number of items pass billions, regardless of the
implementation platform.

Google Architecture

From: Anne & Lynn Wheeler <lynn@garlic.com>
Subject: Re: Google Architecture
Newsgroups: bit.listserv.ibm-main,alt.folklore.computers
CC: ibmmain <ibm-main@bama.ua.edu>
Date: Wed, 07 Jun 2006 08:01:16 -0600

Anne & Lynn Wheeler wrote:

then there is the way-back machine. in a thread looking at UK chip&pin
vulnerabilities, a URL (from a few years ago) that went into some
discussion of the yes card vulnerability, appeared to have become
cloaked. however it was still available at the way-back machine.

re:
http://www.garlic.com/~lynn/2006l.html#27 Google Architecture

for even more drift ... a news item from later yesterday

UK Detects Chip-And-PIN Security Flaw
http://www.cardtechnology.com/article.html?id=20060606I2K75YSX

APACS says the security lapse came to light in a recent study of the
authentication technology used in the UK's new "chip-and-PIN" card system.

... snip ...

and some comment
http://www.garlic.com/~lynn/aadsm23.htm#55 UK Detects Chip-And-PIN Security Flaw

not too long after the exploit (from earlier deployments) being
documented in 2002 ... it was explained to a group from the ATM
industry ... leading somebody in the audience to quip do you
mean that they managed to spend a couple billion dollars to prove that
chips are less secure than magstripes.

Google Architecture

From: Anne & Lynn Wheeler <lynn@garlic.com>
Subject: Re: Google Architecture
Newsgroups: bit.listserv.ibm-main,alt.folklore.computers
Date: Wed, 07 Jun 2006 18:28:58 -0600

Anne & Lynn Wheeler wrote:

for even more drift ... a news item from later yesterday

UK Detects Chip-And-PIN Security Flaw
http://www.cardtechnology.com/article.html?id=20060606I2K75YSX

APACS says the security lapse came to light in a recent study of the
authentication technology used in the UK's new "chip-and-PIN" card system.

... snip ...

and some comment
http://www.garlic.com/~lynn/aadsm23.htm#55 UK Detects Chip-And-PIN Security Flaw

not too long after the exploit (from earlier deployments) being
documented in 2002 ... it was explained to a group from the ATM industry
... leading somebody in the audience to quip do you mean that they
managed to spend a couple billion dollars to prove that chips are less
secure than magstripes.

re:
http://www.garlic.com/~lynn/2006l.html#32 Google Architecture

a little drift back to ibm:
http://www-03.ibm.com/industries/financialservices/doc/content/solution/1026217103.html

from above:

Safeway and its technology partner IBM were involved in the first 'Chip
and Pin' trials held in the UK in 1997. Recently, Safeway engaged IBM
again to provide the Electronic Payment System (EPS) infrastructure in
support of the company's push forward with the introduction of 'Chip and
Pin.'

... snip ...

the 2002 article mentioning yes card vulnerability describes exploit
involving chip&pin deployments in 2002 and earlier.

the most recent article yesterday describes the current chip&pin
deployment apparently with the same vulnerability as described in the
2002 article mentioning yes card exploits.

for the yes card exploits in the 90s and thru (at least) 2002,
technology that had been around for some time involving compromised
and/or counterfeit terminals (that had been havesting magstripe and pins
used for creating counterfeit debit cards) ... was adapted to harvesting
chip&pin "SDA" data.

The harvested "SDA" data was then loaded into a counterfeit chip. The
terminal chip&pin standard called for authenticating the card and then
asking the card 1) if the entered PIN was correct (YES), 2) if the
transaction was to be offline (YES), and 3) if the transaction was
within the account credit limit (YES). The harvested "SDA" data was
sufficient for a counterfeit card to fool a terminal ... and then the
counterfeit card was programmed to always answer YES to all the
terminal's questions (resulting in giving the counterfeit card the yes
card name). It was not actually necessary to harvest the valid PIN
since the counterfeit yes card would always claim any entered PIN was
valid.

Part of the issue with the yes card vulnerability was that it was an
attack on the terminal (not the card). Regardless of how valid chip&pin
cards had been programmed ... once an attacker had havested "SDA" data
... it was able to create counterfeit yes cards (and attack chip&pin
terminals).

somewhat related posting
http://www.garlic.com/~lynn/aadsm23.htm#56 UK Detects Chip-and-PIN Security Flaw

Dual Core CPUs are slower than Dual Single core CPUs ??

From: Anne & Lynn Wheeler <lynn@garlic.com>
Subject: Re: Dual Core CPUs are slower than Dual Single core CPUs ??
Newsgroups: comp.arch,alt.folklore.computers
Date: Thu, 08 Jun 2006 11:12:44 -0600

"Mayank" writes:

I believe, if indpendent single threaded applications need to be run,
then SMP (dual single core processors) shall provide a better
throughput.

CMP (dual core processors) have more than one execution unit on the
same die, possibly sharing L2 cache and FSB. CMP (and so did SMT)
probably came into existance because Instruction Level Parallelism was
not providing further parallelism (with power playing an important
role). If multi-threaded applications need to be run, then CMP in most
cases shall provide better throughput. This is due to inter thread data
sharing which shall hide the memory latencies.

and the multiple threads are conserving cache lines in many cases by
making use of the exact same data (so you may be getting higher per
instruction cache hit ratio for the same number of cache lines).

there is an analogy to this from long ago and far away involving real
storage for tss/360 paging (from the 60s). tss/360 was originally
announced to run on a 512kbyte 360/67 ... but the tss/360 (fixed)
kernel was rapidly growing. eventually the minimum was 768kbytes and
to really get anything done with tss/360 you needed 1024kbytes
(largest memory configuration).

then then benchmarked two processor tss/360 on a two processor 360/67
with two megabytes of real storage (each processor came with 1mbyte
max. and multiprocessor support allowed the addressing to be linear)
... and tss/360 thruput was coming out around 3.5times that of tss/360
uniprocessor operation.

somebody made the claim that tss/360 scale-up, multiprocessor support
and algorithms were obviously the best in the industry ... being able
to get 3.5 times the thruput with only two times the resources.

it turns out that it was a relative measurement, both tss/360
uniprocessor and multiprocessor thruput was quite bad ... using an
absolute measure (as opposed to purely relative measurement).

the issue was that the tss/360 kernel requirements had grown so that
if you attempted to perform almost any operations ... with the amount
of real storage left over for paging in a 1mbyte configuration
... would page thrash. with double the real storage (2mbytes) ... the
amount of real storage left over for application paging increased by a
factor of 5-10 times (compared to single processor, 1mbyte
configuration) ... resulting in tss/360 seeing 3.5 times the aggregate
thruput (in two processor configuration) relative to single processor
configuration (however, neither numbers were actually that
remarkable).

--
Anne & Lynn Wheeler | http://www.garlic.com/~lynn/

Token-ring vs Ethernet - 10 years later

From: Anne & Lynn Wheeler <lynn@garlic.com>
Subject: Re: Token-ring vs Ethernet - 10 years later
Newsgroups: bit.listserv.ibm-main,alt.folklore.computers
Date: Thu, 08 Jun 2006 15:03:23 -0600

Gilbert Saint-Flour wrote:

The last three examples were sponsored (or developed) by IBM, and many
IBM competitors supported the non-IBM solution precisely because it was
that, non-IBM.  In the case of Micro-channel and OS/2, licensing issues
didn't help with PC companies like Compaq and HP.

TR also got a lot of bad press because a single PC could wreak havoc on
the ring simply because it was configured for 4 Mb/s instead of 16
Mb/s, and finding the culprit was sometimes quite a challenge.
Ethernet, of course, had a lot of problems of its own, but it didn't
have this one.

there are a whole bunch of issues.

as part of the SAA terminal emulation strategy,
http://www.garlic.com/~lynn/subnetwork.html#emulation

the T/R cards were built with per adapter thruput targeted at the
terminal emulation market segment (say 300 PCs on the same ring).
austin had designed & built their own 4mbit t/r (16bit isa) for
workstation environments. for rs/6000 they were forced to use the
corporate standard 16mbit microchannel t/r card. this card had lower per
card thruput than the pc/rt 4mbit t/r card (they weren't allowed to do
their own 16mbit microchannel t/r card that had even the same per card
thruput as their 4mbit 16bit ISA t/r card).

as part of moving research up the hill from sjr to alm, the new alm
building had extensive new wiring. however, in detailed tests they were
finding that 10mbit ethernet had higher aggregate thruput and lower
latency over the cat5 wiring than 16mbit t/r going over the same cat5
wiring.

in the SAA time-frame we had come up with 3-tier architecture
http://www.garlic.com/~lynn/subnetwork.html#3tier

and were out pitching it to customer executives .... including examples
showing 10mbit (cat5 wiring) ethernet deployments compared to 16mbit t/r
deployments (using same cat5 wiring).

we were taking lots of heat from SAA forces which were actively trying
to contain 2-tier/client-server and return the paradigm to the terminal
emulation from the first half of the 80s (so you didn't need faster per
card thruput because you were stuck in terminal emulation paradigm and
you were stuck in terminal emulation paradigm because of the limited per
card thruput).

we were also taking lots of heat from the t/r contingent. one of the t/r
centers had published a paper showing 16mbit t/r compared to "ethernet"
... with ethernet degrading to less than 1mbit aggregate effective
thruput. it appeared to be using the ancient 3mbit ethernet
specification which didn't even include listen before transmit (part of
the 10mbit standard).

about the same time, annual acm sigcomm had a paper that did some
detailed look at commoningly deployed ethernet. one of the tests had
30 stations in tight low-level device driver loop transmitting minimum
sized packets as fast as possible. In this scenario, effective
aggregate thruput of 10mbit ethernet dropped off to 8.5mbits from a
normal environment with effective aggregate thruput around 9.5mbits.

disclaimer: my wife is co-inventor for token passing patents (US
and international) from the late 70s.

Token-ring vs Ethernet - 10 years later

From: Anne & Lynn Wheeler <lynn@garlic.com>
Subject: Re: Token-ring vs Ethernet - 10 years later
Newsgroups: bit.listserv.ibm-main,alt.folklore.computers
Date: Thu, 08 Jun 2006 18:42:55 -0600

Charles Mills wrote:

Price and also simplicity of implementation.

Price is especially significant when people are tip-toeing into something
not sure if they are going to like it - that was the case with VHS and Beta.
"I'll get one of these cheap VHS VCRs, and if I like it, I'll get a good
Beta later." Of course, once they had a library of VHS tapes, "later" never
came.

re:
http://www.garlic.com/~lynn/2006l.html#35 Token-ring vs Ethernet - 10 years later

microchannel 16mbit t/r card were going for something like $900/card

aggregate 16mbit t/r LAN had lower aggregate network thruput than 10mbit
ethernet

individual microchannel 16mbit t/r adapter card had lower per card
thruput than almost any enet cards and even lower per card thruput than
the pc/rt 4mbit t/r 16bit isa card.

...................

there were 10mbit 16bit isa ethernet cards with numerous different chips
... intel (82586), amd (lance), and several others ... with list pri