…meher

SQL is a standard language for accessing and manipulating databases.

What is SQL?

• SQL stands for Structured Query Language
• SQL lets you access and manipulate databases
• SQL is an ANSI (American National Standards Institute) standard

What Can SQL do?

• SQL can execute queries against a database
• SQL can retrieve data from a database
• SQL can insert records in a database
• SQL can update records in a database
• SQL can delete records from a database
• SQL can create new databases
• SQL can create new tables in a database
• SQL can create stored procedures in a database
• SQL can create views in a database
• SQL can set permissions on tables, procedures, and views

SQL is a Standard – BUT….

Although SQL is an ANSI (American National Standards Institute) standard, there are many different versions of the SQL language.

However, to be compliant with the ANSI standard, they all support at least the major commands (such as SELECT, UPDATE, DELETE, INSERT, WHERE) in a similar manner.

Note: Most of the SQL database programs also have their own proprietary extensions in addition to the SQL standard!

Using SQL in Your Web Site

To build a web site that shows some data from a database, you will need the following:

• An RDBMS database program (i.e. MS Access, SQL Server, MySQL)
• A server-side scripting language, like PHP or ASP
• SQL
• HTML / CSS

RDBMS

RDBMS stands for Relational Database Management System.

RDBMS is the basis for SQL, and for all modern database systems like MS SQL Server, IBM DB2, Oracle, MySQL, and Microsoft Access.

The data in RDBMS is stored in database objects called tables.

A table is a collections of related data entries and it consists of columns and rows.

SQL Syntax

Database Tables

A database most often contains one or more tables. Each table is identified by a name (e.g. “Customers” or “Orders”). Tables contain records (rows) with data.

Below is an example of a table called “Persons”:

The table above contains three records (one for each person) and five columns (P_Id, LastName, FirstName, Address, and City).

SQL Statements

Most of the actions you need to perform on a database are done with SQL statements.

The following SQL statement will select all the records in the “Persons” table:

In this tutorial we will teach you all about the different SQL statements.

Keep in Mind That…

• SQL is not case sensitive

Semicolon after SQL Statements?

Some database systems require a semicolon at the end of each SQL statement.

Semicolon is the standard way to separate each SQL statement in database systems that allow more than one SQL statement to be executed in the same call to the server.

We are using MS Access and SQL Server 2000 and we do not have to put a semicolon after each SQL statement, but some database programs force you to use it.

SQL DML and DDL

SQL can be divided into two parts: The Data Manipulation Language (DML) and the Data Definition Language (DDL).

The query and update commands form the DML part of SQL:

• SELECT – extracts data from a database
• UPDATE – updates data in a database
• DELETE – deletes data from a database
• INSERT INTO – inserts new data into a database

The DDL part of SQL permits database tables to be created or deleted. It also define indexes (keys), specify links between tables, and impose constraints between tables. The most important DDL statements in SQL are:

• CREATE DATABASE – creates a new database
• ALTER DATABASE – modifies a database
• CREATE TABLE – creates a new table
• ALTER TABLE – modifies a table
• DROP TABLE – deletes a table
• CREATE INDEX – creates an index (search key)
• DROP INDEX – deletes an index

SQL SELECT Statement

This chapter will explain the SELECT and the SELECT * statements.

The SQL SELECT Statement

The SELECT statement is used to select data from a database.

The result is stored in a result table, called the result-set.

SQL SELECT Syntax

and

Note: SQL is not case sensitive. SELECT is the same as select.

An SQL SELECT Example

The “Persons” table:

Now we want to select the content of the columns named “LastName” and “FirstName” from the table above.

We use the following SELECT statement:

The result-set will look like this:

SELECT * Example

Now we want to select all the columns from the “Persons” table.

We use the following SELECT statement: 

Tip: The asterisk (*) is a quick way of selecting all columns!

The result-set will look like this:

Navigation in a Result-set

Most database software systems allow navigation in the result-set with programming functions, like: Move-To-First-Record, Get-Record-Content, Move-To-Next-Record, etc.

Programming functions like these are not a part of this tutorial. To learn about accessing data with function calls, please visit our ADO tutorial or our PHP tutorial.

SQL SELECT DISTINCT Statement

This chapter will explain the SELECT DISTINCT statement.

The SQL SELECT DISTINCT Statement

In a table, some of the columns may contain duplicate values. This is not a problem, however, sometimes you will want to list only the different (distinct) values in a table.

The DISTINCT keyword can be used to return only distinct (different) values.

SQL SELECT DISTINCT Syntax

SELECT DISTINCT Example

The “Persons” table:

Now we want to select only the distinct values from the column named “City” from the table above.

We use the following SELECT statement:

The result-set will look like this:

SQL WHERE Clause

The WHERE clause is used to filter records.

The WHERE Clause 

The WHERE clause is used to extract only those records that fulfill a specified criterion.

SQL WHERE Syntax

WHERE Clause Example

The “Persons” table:

Now we want to select only the persons living in the city “Sandnes” from the table above.

We use the following SELECT statement:

The result-set will look like this:

Quotes Around Text Fields

SQL uses single quotes around text values (most database systems will also accept double quotes).

Although, numeric values should not be enclosed in quotes.

For text values:

For numeric values:

Operators Allowed in the WHERE Clause

With the WHERE clause, the following operators can be used:

Note: In some versions of SQL the <> operator may be written as !=

SQL AND & OR Operators

The AND & OR operators are used to filter records based on more than one condition.

The AND & OR Operators

The AND operator displays a record if both the first condition and the second condition is true.

The OR operator displays a record if either the first condition or the second condition is true.

AND Operator Example

The “Persons” table:

Now we want to select only the persons with the first name equal to “Tove” AND the last name equal to “Svendson”:

We use the following SELECT statement:

The result-set will look like this:

OR Operator Example

Now we want to select only the persons with the first name equal to “Tove” OR the first name equal to “Ola”:

We use the following SELECT statement:

The result-set will look like this:

Combining AND & OR

You can also combine AND and OR (use parenthesis to form complex expressions).

Now we want to select only the persons with the last name equal to “Svendson” AND the first name equal to “Tove” OR to “Ola”:

We use the following SELECT statement:

The result-set will look like this:

SQL ORDER BY Keyword

The ORDER BY keyword is used to sort the result-set.

The ORDER BY Keyword

The ORDER BY keyword is used to sort the result-set by a specified column.

The ORDER BY keyword sort the records in ascending order by default.

If you want to sort the records in a descending order, you can use the DESC keyword.

SQL ORDER BY Syntax

ORDER BY Example

The “Persons” table:

Now we want to select all the persons from the table above, however, we want to sort the persons by their last name.

We use the following SELECT statement:

The result-set will look like this:

ORDER BY DESC Example

Now we want to select all the persons from the table above, however, we want to sort the persons descending by their last name.

We use the following SELECT statement:

The result-set will look like this:

SQL INSERT INTO Statement

The INSERT INTO statement is used to insert new records in a table.

The INSERT INTO Statement

The INSERT INTO statement is used to insert a new row in a table.

SQL INSERT INTO Syntax

It is possible to write the INSERT INTO statement in two forms.

The first form doesn’t specify the column names where the data will be inserted, only their values:

The second form specifies both the column names and the values to be inserted:

SQL INSERT INTO Example

We have the following “Persons” table:

Now we want to insert a new row in the “Persons” table.

We use the following SQL statement:

The “Persons” table will now look like this:

Insert Data Only in Specified Columns

It is also possible to only add data in specific columns.

The following SQL statement will add a new row, but only add data in the “P_Id”, “LastName” and the “FirstName” columns:

The “Persons” table will now look like this:

SQL UPDATE Statement

The UPDATE statement is used to update records in a table.

The UPDATE Statement

The UPDATE statement is used to update existing records in a table.

SQL UPDATE Syntax

Note: Notice the WHERE clause in the UPDATE syntax. The WHERE clause specifies which record or records that should be updated. If you omit the WHERE clause, all records will be updated!

SQL UPDATE Example

The “Persons” table:

Now we want to update the person “Tjessem, Jakob” in the “Persons” table.

We use the following SQL statement:

The “Persons” table will now look like this:

SQL UPDATE Warning

Be careful when updating records. If we had omitted the WHERE clause in the example above, like this:

The “Persons” table would have looked like this:

SQL DELETE Statement

The DELETE statement is used to delete records in a table.

The DELETE Statement

The DELETE statement is used to delete rows in a table.

SQL DELETE Syntax

Note: Notice the WHERE clause in the DELETE syntax. The WHERE clause specifies which record or records that should be deleted. If you omit the WHERE clause, all records will be deleted!

SQL DELETE Example

The “Persons” table:

Now we want to delete the person “Tjessem, Jakob” in the “Persons” table.

We use the following SQL statement:

The “Persons” table will now look like this:

Delete All Rows

It is possible to delete all rows in a table without deleting the table. This means that the table structure, attributes, and indexes will be intact:

Note: Be very careful when deleting records. You cannot undo this statement!

SQL TOP Clause

The TOP Clause

The TOP clause is used to specify the number of records to return.

The TOP clause can be very useful on large tables with thousands of records. Returning a large number of records can impact on performance.

Note: Not all database systems support the TOP clause.

SQL Server Syntax

SQL SELECT TOP Equivalent in MySQL and Oracle

MySQL Syntax

Example

Oracle Syntax

Example

SQL TOP Example

The “Persons” table:

Now we want to select only the two first records in the table above.

We use the following SELECT statement:

The result-set will look like this:

SQL TOP PERCENT Example

The “Persons” table:

Now we want to select only 50% of the records in the table above.

We use the following SELECT statement:

The result-set will look like this:

SQL LIKE Operator

The LIKE operator is used in a WHERE clause to search for a specified pattern in a column.

The LIKE Operator

The LIKE operator is used to search for a specified pattern in a column.

SQL LIKE Syntax

LIKE Operator Example

The “Persons” table:

Now we want to select the persons living in a city that starts with “s” from the table above.

We use the following SELECT statement:

The “%” sign can be used to define wildcards (missing letters in the pattern) both before and after the pattern.

The result-set will look like this:

Next, we want to select the persons living in a city that ends with an “s” from the “Persons” table.

We use the following SELECT statement:

The result-set will look like this:

Next, we want to select the persons living in a city that contains the pattern “tav” from the “Persons” table.

We use the following SELECT statement:

The result-set will look like this:

It is also possible to select the persons living in a city that NOT contains the pattern “tav” from the “Persons” table, by using the NOT keyword.

We use the following SELECT statement:

The result-set will look like this:

SQL Wildcards

SQL wildcards can be used when searching for data in a database.

SQL Wildcards 

SQL wildcards can substitute for one or more characters when searching for data in a database.

SQL wildcards must be used with the SQL LIKE operator.

With SQL, the following wildcards can be used:

SQL Wildcard Examples

We have the following “Persons” table:

Using the % Wildcard

Now we want to select the persons living in a city that starts with “sa” from the “Persons” table.

We use the following SELECT statement:

The result-set will look like this:

Next, we want to select the persons living in a city that contains the pattern “nes” from the “Persons” table.

We use the following SELECT statement:

The result-set will look like this:

Using the _ Wildcard

Now we want to select the persons with a first name that starts with any character, followed by “la” from the “Persons” table.

We use the following SELECT statement:

The result-set will look like this:

Next, we want to select the persons with a last name that starts with “S”, followed by any character, followed by “end”, followed by any character, followed by “on” from the “Persons” table.

We use the following SELECT statement:

The result-set will look like this:

Using the [charlist] Wildcard

Now we want to select the persons with a last name that starts with “b” or “s” or “p” from the “Persons” table.

We use the following SELECT statement:

The result-set will look like this:

Next, we want to select the persons with a last name that do not start with “b” or “s” or “p” from the “Persons” table.

We use the following SELECT statement:

The result-set will look like this:

SQL IN Operator

The IN Operator

The IN operator allows you to specify multiple values in a WHERE clause.

SQL IN Syntax

IN Operator Example

The “Persons” table:

Now we want to select the persons with a last name equal to “Hansen” or “Pettersen” from the table above.

We use the following SELECT statement:

The result-set will look like this:

SQL BETWEEN Operator

The BETWEEN operator is used in a WHERE clause to select a range of data between two values.

The BETWEEN Operator

The BETWEEN operator selects a range of data between two values. The values can be numbers, text, or dates.

SQL BETWEEN Syntax

BETWEEN Operator Example

The “Persons” table:

Now we want to select the persons with a last name alphabetically between “Hansen” and “Pettersen” from the table above.

We use the following SELECT statement:

The result-set will look like this:

Note: The BETWEEN operator is treated differently in different databases.

In some databases, persons with the LastName of “Hansen” or “Pettersen” will not be listed, because the BETWEEN operator only selects fields that are between and excluding the test values).

In other databases, persons with the LastName of “Hansen” or “Pettersen” will be listed, because the BETWEEN operator selects fields that are between and including the test values).

And in other databases, persons with the LastName of “Hansen” will be listed, but “Pettersen” will not be listed (like the example above), because the BETWEEN operator selects fields between the test values, including the first test value and excluding the last test value.

Therefore: Check how your database treats the BETWEEN operator.

Example 2

To display the persons outside the range in the previous example, use NOT BETWEEN:

The result-set will look like this:

SQL Alias

With SQL, an alias name can be given to a table or to a column.

SQL Alias

You can give a table or a column another name by using an alias. This can be a good thing to do if you have very long or complex table names or column names.

An alias name could be anything, but usually it is short.

SQL Alias Syntax for Tables

SQL Alias Syntax for Columns

Alias Example

Assume we have a table called “Persons” and another table called “Product_Orders”. We will give the table aliases of “p” an “po” respectively.

Now we want to list all the orders that “Ola Hansen” is responsible for.

We use the following SELECT statement:

The same SELECT statement without aliases:

As you’ll see from the two SELECT statements above; aliases can make queries easier to both write and to read.

SQL Joins

SQL joins are used to query data from two or more tables, based on a relationship between certain columns in these tables.

SQL JOIN

The JOIN keyword is used in an SQL statement to query data from two or more tables, based on a relationship between certain columns in these tables.

Tables in a database are often related to each other with keys.

A primary key is a column (or a combination of columns) with a unique value for each row. Each primary key value must be unique within the table. The purpose is to bind data together, across tables, without repeating all of the data in every table.

Look at the “Persons” table:

Note that the “P_Id” column is the primary key in the “Persons” table. This means that no two rows can have the same P_Id. The P_Id distinguishes two persons even if they have the same name.

Next, we have the “Orders” table:

Note that the “O_Id” column is the primary key in the “Orders” table and that the “P_Id” column refers to the persons in the “Persons” table without using their names.

Notice that the relationship between the two tables above is the “P_Id” column.

Different SQL JOINs

Before we continue with examples, we will list the types of JOIN you can use, and the differences between them.

• JOIN: Return rows when there is at least one match in both tables
• LEFT JOIN: Return all rows from the left table, even if there are no matches in the right table
• RIGHT JOIN: Return all rows from the right table, even if there are no matches in the left table
• FULL JOIN: Return rows when there is a match in one of the tables

SQL INNER JOIN Keyword

SQL INNER JOIN Keyword

The INNER JOIN keyword return rows when there is at least one match in both tables.

SQL INNER JOIN Syntax

PS: INNER JOIN is the same as JOIN.

SQL INNER JOIN Example

The “Persons” table:

The “Orders” table:

Now we want to list all the persons with any orders.

We use the following SELECT statement:

The result-set will look like this:

The INNER JOIN keyword return rows when there is at least one match in both tables. If there are rows in “Persons” that do not have matches in “Orders”, those rows will NOT be listed.

SQL LEFT JOIN Keyword

SQL LEFT JOIN Keyword

The LEFT JOIN keyword returns all rows from the left table (table_name1), even if there are no matches in the right table (table_name2).

SQL LEFT JOIN Syntax

PS: In some databases LEFT JOIN is called LEFT OUTER JOIN.

SQL LEFT JOIN Example

The “Persons” table:

The “Orders” table:

Now we want to list all the persons and their orders – if any, from the tables above.

We use the following SELECT statement:

The result-set will look like this:

The LEFT JOIN keyword returns all the rows from the left table (Persons), even if there are no matches in the right table (Orders).

SQL RIGHT JOIN Keyword

SQL RIGHT JOIN Keyword

The RIGHT JOIN keyword Return all rows from the right table (table_name2), even if there are no matches in the left table (table_name1).

SQL RIGHT JOIN Syntax

PS: In some databases RIGHT JOIN is called RIGHT OUTER JOIN.

SQL RIGHT JOIN Example

The “Persons” table:

The “Orders” table:

Now we want to list all the orders with containing persons – if any, from the tables above.

We use the following SELECT statement:

The result-set will look like this:

The RIGHT JOIN keyword returns all the rows from the right table (Orders), even if there are no matches in the left table (Persons).

SQL FULL JOIN Keyword

SQL FULL JOIN Keyword

The FULL JOIN keyword return rows when there is a match in one of the tables.

SQL FULL JOIN Syntax

SQL FULL JOIN Example

The “Persons” table:

The “Orders” table:

Now we want to list all the persons and their orders, and all the orders with their persons.

We use the following SELECT statement:

The result-set will look like this:

The FULL JOIN keyword returns all the rows from the left table (Persons), and all the rows from the right table (Orders). If there are rows in “Persons” that do not have matches in “Orders”, or if there are rows in “Orders” that do not have matches in “Persons”, those rows will be listed as well.

SQL UNION Operator

The SQL UNION operator combines two or more SELECT statements.

The SQL UNION Operator

The UNION operator is used to combine the result-set of two or more SELECT statements.

Notice that each SELECT statement within theUNIONmust have the same number of columns. The columns must also have similar data types. Also, the columns in each SELECT statement must be in the same order.

SQL UNION Syntax

Note: The UNION operator selects only distinct values by default. To allow duplicate values, use UNION ALL.

SQLUNIONALL Syntax

PS: The column names in the result-set of a UNION are always equal to the column names in the first SELECT statement in theUNION.

SQL UNION Example

Look at the following tables:

“Employees_Norway”:

“Employees_USA”:

Now we want to list all the different employees inNorway andUSA.

We use the following SELECT statement:

The result-set will look like this:

Note: This command cannot be used to list all employees inNorway andUSA. In the example above we have two employees with equal names, and only one of them will be listed. The UNION command selects only distinct values.

SQLUNIONALL Example

Now we want to list all employees inNorway andUSA:

Result

SQL SELECT INTO Statement

The SQL SELECT INTO statement can be used to create backup copies of tables.

The SQL SELECT INTO Statement

The SELECT INTO statement selects data from one table and inserts it into a different table.

The SELECT INTO statement is most often used to create backup copies of tables.

SQL SELECT INTO Syntax

We can select all columns into the new table:

Or we can select only the columns we want into the new table:

SQL SELECT INTO Example

Make a Backup Copy – Now we want to make an exact copy of the data in our “Persons” table.

We use the following SQL statement:

We can also use the IN clause to copy the table into another database:

We can also copy only a few fields into the new table:

SQL SELECT INTO – With a WHERE Clause

We can also add a WHERE clause.

The following SQL statement creates a “Persons_Backup” table with only the persons who lives in the city “Sandnes”:

SQL SELECT INTO – Joined Tables

Selecting data from more than one table is also possible.

The following example creates a “Persons_Order_Backup” table contains data from the two tables “Persons” and “Orders”:

SQL CREATE DATABASE Statement

The CREATE DATABASE Statement

The CREATE DATABASE statement is used to create a database.

SQL CREATE DATABASE Syntax

CREATE DATABASE Example

Now we want to create a database called “my_db”.

We use the following CREATE DATABASE statement:

Database tables can be added with the CREATE TABLE statement.

SQL CREATE TABLE Statement

The CREATE TABLE Statement

The CREATE TABLE statement is used to create a table in a database.

SQL CREATE TABLE Syntax

The data type specifies what type of data the column can hold. For a complete reference of all the data types available in MS Access, MySQL, and SQL Server, go to our complete Data Types reference.

CREATE TABLE Example

Now we want to create a table called “Persons” that contains five columns: P_Id, LastName, FirstName, Address, and City.

We use the following CREATE TABLE statement:

The P_Id column is of type int and will hold a number. The LastName, FirstName, Address, and City columns are of type varchar with a maximum length of 255 characters.

The empty “Persons” table will now look like this:

The empty table can be filled with data with the INSERT INTO statement.

SQL Constraints

SQL Constraints

Constraints are used to limit the type of data that can go into a table.

Constraints can be specified when a table is created (with the CREATE TABLE statement) or after the table is created (with the ALTER TABLE statement).

We will focus on the following constraints:

• NOT NULL
• UNIQUE
• PRIMARY KEY
• FOREIGN KEY
• CHECK
• DEFAULT

The next chapters will describe each constraint in details.

SQL NOT NULL Constraint

SQL NOT NULL Constraint

The NOT NULL constraint enforces a column to NOT accept NULL values.

The NOT NULL constraint enforces a field to always contain a value. This means that you cannot insert a new record, or update a record without adding a value to this field.

The following SQL enforces the “P_Id” column and the “LastName” column to not accept NULL values:

SQL UNIQUE Constraint

SQL UNIQUE Constraint

The UNIQUE constraint uniquely identifies each record in a database table.

The UNIQUE and PRIMARY KEY constraints both provide a guarantee for uniqueness for a column or set of columns.

A PRIMARY KEY constraint automatically has a UNIQUE constraint defined on it.

Note that you can have many UNIQUE constraints per table, but only one PRIMARY KEY constraint per table.

SQL UNIQUE Constraint on CREATE TABLE

The following SQL creates a UNIQUE constraint on the “P_Id” column when the “Persons” table is created:

MySQL:

SQL Server / Oracle / MS Access:

To allow naming of a UNIQUE constraint, and for defining a UNIQUE constraint on multiple columns, use the following SQL syntax:

MySQL / SQL Server / Oracle / MS Access:

SQL UNIQUE Constraint on ALTER TABLE

To create a UNIQUE constraint on the “P_Id” column when the table is already created, use the following SQL:

MySQL / SQL Server / Oracle / MS Access:

To allow naming of a UNIQUE constraint, and for defining a UNIQUE constraint on multiple columns, use the following SQL syntax:

MySQL / SQL Server / Oracle / MS Access:

To DROP a UNIQUE Constraint

To drop a UNIQUE constraint, use the following SQL:

MySQL:

SQL Server / Oracle / MS Access:

SQL PRIMARY KEY Constraint

SQL PRIMARY KEY Constraint

The PRIMARY KEY constraint uniquely identifies each record in a database table.

Primary keys must contain unique values.

A primary key column cannot contain NULL values.

Each table should have a primary key, and each table can have only one primary key.

SQL PRIMARY KEY Constraint on CREATE TABLE

The following SQL creates a PRIMARY KEY on the “P_Id” column when the “Persons” table is created:

MySQL:

SQL Server / Oracle / MS Access:

To allow naming of a PRIMARY KEY constraint, and for defining a PRIMARY KEY constraint on multiple columns, use the following SQL syntax:

MySQL / SQL Server / Oracle / MS Access:

SQL PRIMARY KEY Constraint on ALTER TABLE

To create a PRIMARY KEY constraint on the “P_Id” column when the table is already created, use the following SQL:

MySQL / SQL Server / Oracle / MS Access:

To allow naming of a PRIMARY KEY constraint, and for defining a PRIMARY KEY constraint on multiple columns, use the following SQL syntax:

MySQL / SQL Server / Oracle / MS Access:

Note: If you use the ALTER TABLE statement to add a primary key, the primary key column(s) must already have been declared to not contain NULL values (when the table was first created).

To DROP a PRIMARY KEY Constraint

To drop a PRIMARY KEY constraint, use the following SQL:

MySQL:

SQL Server / Oracle / MS Access:

SQL FOREIGN KEY Constraint

SQL FOREIGN KEY Constraint

A FOREIGN KEY in one table points to a PRIMARY KEY in another table.

Let’s illustrate the foreign key with an example. Look at the following two tables:

The “Persons” table:

The “Orders” table:

Note that the “P_Id” column in the “Orders” table points to the “P_Id” column in the “Persons” table.

The “P_Id” column in the “Persons” table is the PRIMARY KEY in the “Persons” table.

The “P_Id” column in the “Orders” table is a FOREIGN KEY in the “Orders” table.

The FOREIGN KEY constraint is used to prevent actions that would destroy link between tables.

The FOREIGN KEY constraint also prevents that invalid data is inserted into the foreign key column, because it has to be one of the values contained in the table it points to.

SQL FOREIGN KEY Constraint on CREATE TABLE

The following SQL creates a FOREIGN KEY on the “P_Id” column when the “Orders” table is created:

MySQL:

SQL Server / Oracle / MS Access:

To allow naming of a FOREIGN KEY constraint, and for defining a FOREIGN KEY constraint on multiple columns, use the following SQL syntax:

MySQL / SQL Server / Oracle / MS Access:

SQL FOREIGN KEY Constraint on ALTER TABLE

To create a FOREIGN KEY constraint on the “P_Id” column when the “Orders” table is already created, use the following SQL:

MySQL / SQL Server / Oracle / MS Access:

To allow naming of a FOREIGN KEY constraint, and for defining a FOREIGN KEY constraint on multiple columns, use the following SQL syntax:

MySQL / SQL Server / Oracle / MS Access:

To DROP a FOREIGN KEY Constraint

To drop a FOREIGN KEY constraint, use the following SQL:

MySQL:

SQL Server / Oracle / MS Access:

SQL CHECK Constraint

SQL CHECK Constraint

The CHECK constraint is used to limit the value range that can be placed in a column.

If you define a CHECK constraint on a single column it allows only certain values for this column.

If you define a CHECK constraint on a table it can limit the values in certain columns based on values in other columns in the row.

SQL CHECK Constraint on CREATE TABLE

The following SQL creates a CHECK constraint on the “P_Id” column when the “Persons” table is created. The CHECK constraint specifies that the column “P_Id” must only include integers greater than 0.

My SQL:

SQL Server / Oracle / MS Access:

To allow naming of a CHECK constraint, and for defining a CHECK constraint on multiple columns, use the following SQL syntax:

MySQL / SQL Server / Oracle / MS Access:

SQL CHECK Constraint on ALTER TABLE

To create a CHECK constraint on the “P_Id” column when the table is already created, use the following SQL:

MySQL / SQL Server / Oracle / MS Access:

To allow naming of a CHECK constraint, and for defining a CHECK constraint on multiple columns, use the following SQL syntax:

MySQL / SQL Server / Oracle / MS Access:

To DROP a CHECK Constraint

To drop a CHECK constraint, use the following SQL:

SQL Server / Oracle / MS Access:

SQL DEFAULT Constraint

SQL DEFAULT Constraint

The DEFAULT constraint is used to insert a default value into a column.

The default value will be added to all new records, if no other value is specified.

SQL DEFAULT Constraint on CREATE TABLE

The following SQL creates a DEFAULT constraint on the “City” column when the “Persons” table is created:

My SQL / SQL Server / Oracle / MS Access:

The DEFAULT constraint can also be used to insert system values, by using functions like GETDATE():

SQL DEFAULT Constraint on ALTER TABLE

To create a DEFAULT constraint on the “City” column when the table is already created, use the following SQL:

MySQL:

SQL Server / Oracle / MS Access:

To DROP a DEFAULT Constraint

To drop a DEFAULT constraint, use the following SQL:

MySQL:

SQL Server / Oracle / MS Access:

SQL CREATE INDEX Statement

The CREATE INDEX statement is used to create indexes in tables.

Indexes allow the database application to find data fast; without reading the whole table.

Indexes

An index can be created in a table to find data more quickly and efficiently.

The users cannot see the indexes, they are just used to speed up searches/queries.

Note: Updating a table with indexes takes more time than updating a table without (because the indexes also need an update). So you should only create indexes on columns (and tables) that will be frequently searched against.

SQL CREATE INDEX Syntax

Creates an index on a table. Duplicate values are allowed:

SQL CREATE UNIQUE INDEX Syntax

Creates a unique index on a table. Duplicate values are not allowed:

Note: The syntax for creating indexes varies amongst different databases. Therefore: Check the syntax for creating indexes in your database.

CREATE INDEX Example

The SQL statement below creates an index named “PIndex” on the “LastName” column in the “Persons” table:

If you want to create an index on a combination of columns, you can list the column names within the parentheses, separated by commas:

SQL DROP INDEX, DROP TABLE, and DROP DATABASE

Indexes, tables, and databases can easily be deleted/removed with the DROP statement.

The DROP INDEX Statement

The DROP INDEX statement is used to delete an index in a table.

DROP INDEX Syntax for MS Access:

DROP INDEX Syntax for MS SQL Server:

DROP INDEX Syntax for DB2/Oracle:

DROP INDEX Syntax for MySQL:

The DROP TABLE Statement

The DROP TABLE statement is used to delete a table.

The DROP DATABASE Statement

The DROP DATABASE statement is used to delete a database.

The TRUNCATE TABLE Statement

What if we only want to delete the data inside the table, and not the table itself?

Then, use the TRUNCATE TABLE statement:

SQL ALTER TABLE Statement

The ALTER TABLE Statement

The ALTER TABLE statement is used to add, delete, or modify columns in an existing table.

SQL ALTER TABLE Syntax

To add a column in a table, use the following syntax:

To delete a column in a table, use the following syntax (notice that some database systems don’t allow deleting a column):

To change the data type of a column in a table, use the following syntax:

SQL ALTER TABLE Example

Look at the “Persons” table:

Now we want to add a column named “DateOfBirth” in the “Persons” table.

We use the following SQL statement:

Notice that the new column, “DateOfBirth”, is of type date and is going to hold a date. The data type specifies what type of data the column can hold. For a complete reference of all the data types available in MS Access, MySQL, and SQL Server, go to our complete Data Types reference.

The “Persons” table will now like this:

Change Data Type Example

Now we want to change the data type of the column named “DateOfBirth” in the “Persons” table.

We use the following SQL statement:

Notice that the “DateOfBirth” column is now of type year and is going to hold a year in a two-digit or four-digit format.

DROP COLUMN Example

Next, we want to delete the column named “DateOfBirth” in the “Persons” table.

We use the following SQL statement:

The “Persons” table will now like this:

SQL AUTO INCREMENT Field

Auto-increment allows a unique number to be generated when a new record is inserted into a table.

AUTO INCREMENT a Field

Very often we would like the value of the primary key field to be created automatically every time a new record is inserted.

We would like to create an auto-increment field in a table.

Syntax for MySQL

The following SQL statement defines the “P_Id” column to be an auto-increment primary key field in the “Persons” table:

MySQL uses the AUTO_INCREMENT keyword to perform an auto-increment feature.

By default, the starting value for AUTO_INCREMENT is 1, and it will increment by 1 for each new record.

To let the AUTO_INCREMENT sequence start with another value, use the following SQL statement:

To insert a new record into the “Persons” table, we will not have to specify a value for the “P_Id” column (a unique value will be added automatically):

The SQL statement above would insert a new record into the “Persons” table. The “P_Id” column would be assigned a unique value. The “FirstName” column would be set to “Lars” and the “LastName” column would be set to “Monsen”.

Syntax for SQL Server

The following SQL statement defines the “P_Id” column to be an auto-increment primary key field in the “Persons” table:

The MS SQL Server uses the IDENTITY keyword to perform an auto-increment feature.

By default, the starting value for IDENTITY is 1, and it will increment by 1 for each new record.

To specify that the “P_Id” column should start at value 10 and increment by 5, change the identity to IDENTITY(10,5).

To insert a new record into the “Persons” table, we will not have to specify a value for the “P_Id” column (a unique value will be added automatically):

The SQL statement above would insert a new record into the “Persons” table. The “P_Id” column would be assigned a unique value. The “FirstName” column would be set to “Lars” and the “LastName” column would be set to “Monsen”.

Syntax for Access

The following SQL statement defines the “P_Id” column to be an auto-increment primary key field in the “Persons” table:

The MS Access uses the AUTOINCREMENT keyword to perform an auto-increment feature.

By default, the starting value for AUTOINCREMENT is 1, and it will increment by 1 for each new record.

To specify that the “P_Id” column should start at value 10 and increment by 5, change the autoincrement to AUTOINCREMENT(10,5).

To insert a new record into the “Persons” table, we will not have to specify a value for the “P_Id” column (a unique value will be added automatically):

The SQL statement above would insert a new record into the “Persons” table. The “P_Id” column would be assigned a unique value. The “FirstName” column would be set to “Lars” and the “LastName” column would be set to “Monsen”.

Syntax for Oracle

In Oracle the code is a little bit more tricky.

You will have to create an auto-increment field with the sequence object (this object generates a number sequence).

Use the following CREATE SEQUENCE syntax:

The code above creates a sequence object called seq_person, that starts with 1 and will increment by 1. It will also cache up to 10 values for performance. The cache option specifies how many sequence values will be stored in memory for faster access.

To insert a new record into the “Persons” table, we will have to use the nextval function (this function retrieves the next value from seq_person sequence):

The SQL statement above would insert a new record into the “Persons” table. The “P_Id” column would be assigned the next number from the seq_person sequence. The “FirstName” column would be set to “Lars” and the “LastName” column would be set to “Monsen”.

SQL Date Functions

SQL Dates

The most difficult part when working with dates is to be sure that the format of the date you are trying to insert, matches the format of the date column in the database.

As long as your data contains only the date portion, your queries will work as expected. However, if a time portion is involved, it gets complicated.

Before talking about the complications of querying for dates, we will look at the most important built-in functions for working with dates.

MySQL Date Functions

The following table lists the most important built-in date functions in MySQL:

SQL Server Date Functions

The following table lists the most important built-in date functions in SQL Server:

SQL Date Data Types

MySQL comes with the following data types for storing a date or a date/time value in the database:

• DATE – format YYYY-MM-DD
• DATETIME – format: YYYY-MM-DD HH:MM:SS
• TIMESTAMP – format: YYYY-MM-DD HH:MM:SS
• YEAR – format YYYY or YY

SQL Server comes with the following data types for storing a date or a date/time value in the database:

• DATE – format YYYY-MM-DD
• DATETIME – format: YYYY-MM-DD HH:MM:SS
• SMALLDATETIME – format: YYYY-MM-DD HH:MM:SS
• TIMESTAMP – format: a unique number

Note: The date types are chosen for a column when you create a new table in your database!

For an overview of all data types available, go to our complete Data Types reference.

SQL Working with Dates

You can compare two dates easily if there is no time component involved!

Assume we have the following “Orders” table:

Now we want to select the records with an OrderDate of “2008-11-11″ from the table above.

We use the following SELECT statement:

The result-set will look like this:

Now, assume that the “Orders” table looks like this (notice the time component in the “OrderDate” column):

If we use the same SELECT statement as above:

we will get no result! This is because the query is looking only for dates with no time portion.

Tip: If you want to keep your queries simple and easy to maintain, do not allow time components in your dates!

SQL NULL Values

NULL values represent missing unknown data.

By default, a table column can hold NULL values.

This chapter will explain the IS NULL and IS NOT NULL operators.

SQL NULL Values

If a column in a table is optional, we can insert a new record or update an existing record without adding a value to this column. This means that the field will be saved with a NULL value.

NULL values are treated differently from other values.

NULL is used as a placeholder for unknown or inapplicable values.

Note: It is not possible to compare NULL and 0; they are not equivalent.

SQL Working with NULL Values

Look at the following “Persons” table:

Suppose that the “Address” column in the “Persons” table is optional. This means that if we insert a record with no value for the “Address” column, the “Address” column will be saved with a NULL value.

How can we test for NULL values?

It is not possible to test for NULL values with comparison operators, such as =, <, or <>.

We will have to use the IS NULL and IS NOT NULL operators instead.

SQL IS NULL

How do we select only the records with NULL values in the “Address” column?

We will have to use the IS NULL operator:

The result-set will look like this:

Tip: Always use IS NULL to look for NULL values.

SQL IS NOT NULL

How do we select only the records with no NULL values in the “Address” column?

We will have to use the IS NOT NULL operator:

The result-set will look like this:

In the next chapter we will look at the ISNULL(), NVL(), IFNULL() and COALESCE() functions.

SQL NULL Functions

SQL ISNULL(), NVL(), IFNULL() and COALESCE() Functions

Look at the following “Products” table:

Suppose that the “UnitsOnOrder” column is optional, and may contain NULL values.

We have the following SELECT statement:

In the example above, if any of the “UnitsOnOrder” values are NULL, the result is NULL.

Microsoft’s ISNULL() function is used to specify how we want to treat NULL values.

The NVL(), IFNULL(), and COALESCE() functions can also be used to achieve the same result.

In this case we want NULL values to be zero.

Below, if “UnitsOnOrder” is NULL it will not harm the calculation, because ISNULL() returns a zero if the value is NULL:

SQL Server / MS Access

Oracle

Oracle does not have an ISNULL() function. However, we can use the NVL() function to achieve the same result:

MySQL

MySQL does have an ISNULL() function. However, it works a little bit different from Microsoft’s ISNULL() function.

In MySQL we can use the IFNULL() function, like this:

or we can use the COALESCE() function, like this:

SQL Data Types

Data types and ranges for Microsoft Access, MySQL and SQL Server.

Microsoft Access Data Types

MySQL Data Types

In MySQL there are three main types : text, number, and Date/Time types.

Text types:

Number types:

*The integer types have an extra option called UNSIGNED. Normally, the integer goes from an negative to positive value. Adding the UNSIGNED attribute will move that range up so it starts at zero instead of a negative number.

Date types:

*Even if DATETIME and TIMESTAMP return the same format, they work very differently. In an INSERT or UPDATE query, the TIMESTAMP automatically set itself to the current date and time. TIMESTAMP also accepts various formats, like YYYYMMDDHHMMSS, YYMMDDHHMMSS, YYYYMMDD, or YYMMDD.

SQL Server Data Types

Character strings:

Unicode strings:

Binary types:

Number types:

Date types:

Other data types:

SQL Functions

SQL has many built-in functions for performing calculations on data.

SQL Aggregate Functions

SQL aggregate functions return a single value, calculated from values in a column.

Useful aggregate functions:

• AVG() – Returns the average value
• COUNT() – Returns the number of rows
• FIRST() – Returns the first value
• LAST() – Returns the last value
• MAX() – Returns the largest value
• MIN() – Returns the smallest value
• SUM() – Returns the sum

SQL Scalar functions

SQL scalar functions return a single value, based on the input value.

Useful scalar functions:

• UCASE() – Converts a field to upper case
• LCASE() – Converts a field to lower case
• MID() – Extract characters from a text field
• LEN() – Returns the length of a text field
• ROUND() – Rounds a numeric field to the number of decimals specified
• NOW() – Returns the current system date and time
• FORMAT() – Formats how a field is to be displayed

Tip: The aggregate functions and the scalar functions will be explained in details in the next chapters.

SQL COUNT() Function

The COUNT() function returns the number of rows that matches a specified criteria.

SQL COUNT(column_name) Syntax

The COUNT(column_name) function returns the number of values (NULL values will not be counted) of the specified column:

SQL COUNT(*) Syntax

The COUNT(*) function returns the number of records in a table:

SQL COUNT(DISTINCT column_name) Syntax

The COUNT(DISTINCT column_name) function returns the number of distinct values of the specified column:

Note: COUNT(DISTINCT) works with ORACLE and Microsoft SQL Server, but not with Microsoft Access.

SQL COUNT(column_name) Example

We have the following “Orders” table:

Now we want to count the number of orders from “Customer Nilsen”.

We use the following SQL statement:

The result of the SQL statement above will be 2, because the customer Nilsen has made 2 orders in total:

SQL COUNT(*) Example

If we omit the WHERE clause, like this:

The result-set will look like this:

which is the total number of rows in the table.

SQL COUNT(DISTINCT column_name) Example

Now we want to count the number of unique customers in the “Orders” table.

We use the following SQL statement:

The result-set will look like this:

which is the number of unique customers (Hansen, Nilsen, and Jensen) in the “Orders” table.

SQL LAST() Function

The LAST() Function

The LAST() function returns the last value of the selected column.

SQL LAST() Syntax

SQL LAST() Example

We have the following “Orders” table:

Now we want to find the last value of the “OrderPrice” column.

We use the following SQL statement:

The result-set will look like this:

SQL MIN() Function

The MIN() Function

The MIN() function returns the smallest value of the selected column.

SQL MIN() Syntax

SQL MIN() Example

We have the following “Orders” table:

Now we want to find the smallest value of the “OrderPrice” column.

We use the following SQL statement:

The result-set will look like this:

SQL SUM() Function

The SUM() Function

The SUM() function returns the total sum of a numeric column.

SQL SUM() Syntax

SQL SUM() Example

We have the following “Orders” table:

Now we want to find the sum of all “OrderPrice” fields”.

We use the following SQL statement:

The result-set will look like this:

SQL GROUP BY Statement

Aggregate functions often need an added GROUP BY statement.

The GROUP BY Statement

The GROUP BY statement is used in conjunction with the aggregate functions to group the result-set by one or more columns.

SQL GROUP BY Syntax

SQL GROUP BY Example

We have the following “Orders” table:

Now we want to find the total sum (total order) of each customer.

We will have to use the GROUP BY statement to group the customers.

We use the following SQL statement:

The result-set will look like this:

Nice! Isn’t it?

Let’s see what happens if we omit the GROUP BY statement:

The result-set will look like this:

The result-set above is not what we wanted.

Explanation of why the above SELECT statement cannot be used: The SELECT statement above has two columns specified (Customer and SUM(OrderPrice). The “SUM(OrderPrice)” returns a single value (that is the total sum of the “OrderPrice” column), while “Customer” returns 6 values (one value for each row in the “Orders” table). This will therefore not give us the correct result. However, you have seen that the GROUP BY statement solves this problem.

GROUP BY More Than One Column

We can also use the GROUP BY statement on more than one column, like this:

SQL UCASE() Function

The UCASE() Function

The UCASE() function converts the value of a field to uppercase.

SQL UCASE() Syntax

Syntax for SQL Server

SQL UCASE() Example

We have the following “Persons” table:

Now we want to select the content of the “LastName” and “FirstName” columns above, and convert the “LastName” column to uppercase.

We use the following SELECT statement:

The result-set will look like this:

SQL LCASE() Function

The LCASE() Function

The LCASE() function converts the value of a field to lowercase.

SQL LCASE() Syntax

Syntax for SQL Server

SQL LCASE() Example

We have the following “Persons” table:

Now we want to select the content of the “LastName” and “FirstName” columns above, and convert the “LastName” column to lowercase.

We use the following SELECT statement:

The result-set will look like this:

SQL MID() Function

The MID() Function

The MID() function is used to extract characters from a text field.

SQL MID() Syntax

SQL MID() Example

We have the following “Persons” table:

Now we want to extract the first four characters of the “City” column above.

We use the following SELECT statement:

The result-set will look like this:

SQL LEN() Function

The LEN() Function

The LEN() function returns the length of the value in a text field.

SQL LEN() Syntax

SQL LEN() Example

We have the following “Persons” table:

Now we want to select the length of the values in the “Address” column above.

We use the following SELECT statement:

The result-set will look like this:

SQL ROUND() Function

The ROUND() Function

The ROUND() function is used to round a numeric field to the number of decimals specified.

SQL ROUND() Syntax

SQL ROUND() Example

We have the following “Products” table:

Now we want to display the product name and the price rounded to the nearest integer.

We use the following SELECT statement:

The result-set will look like this:

SQL NOW() Function

The NOW() Function

The NOW() function returns the current system date and time.

SQL NOW() Syntax

SQL NOW() Example

We have the following “Products” table:

Now we want to display the products and prices per today’s date.

We use the following SELECT statement:

The result-set will look like this:

SQL FORMAT() Function

The FORMAT() Function

The FORMAT() function is used to format how a field is to be displayed.

SQL FORMAT() Syntax

SQL FORMAT() Example

We have the following “Products” table:

Now we want to display the products and prices per today’s date (with today’s date displayed in the following format “YYYY-MM-DD”).

We use the following SELECT statement:

The result-set will look like this:

SQL Quick Reference

 ………………………………………………………………………………………………………………………………………………………

Virtual Memory

..meher

1.Background

2.Demand Paging

3.Copy-on-Write

4.Page Replacement

5.Allocation of Frames

6.Thrashing

7.Memory-Mapped Files

8.Allocating Kernel Memory

9.Other Considerations

10.Operating-System Examples

 

 

Objectives

 

 

1.To describe the benefits of a virtual memory system

2.To explain the concepts of demand paging, page-replacement algorithms, and allocation of page frames

3.To discuss the principle of the working-set model

 

 

Background

 

 

1.Virtual memory – separation of user logical memory from physical memory.

1.Only part of the program needs to be in memory for execution

2.Logical address space can therefore be much larger than physical address space

3.Allows address spaces to be shared by several processes

4.Allows for more efficient process creation

2.Virtual memory can be implemented via:

1.Demand paging

2.Demand segmentation

 

 

Virtual Memory That is Larger Than Physical Memory

 

 

Virtual-address Space

 

 

Shared Library Using Virtual Memory

 

 

 

Demand Paging

 

 

1.Bring a page into memory only when it is needed

1.Less I/O needed

2.Less memory needed

3.Faster response

4.More users

2.Page is needed Þ reference to it

1.invalid reference Þ abort

2.not-in-memory Þ bring to memory

3.Lazy swapper – never swaps a page into memory unless page will be needed

1.Swapper that deals with pages is a pager

 

 

Transfer of a Paged Memory to Contiguous Disk Space

 

 

Valid-Invalid Bit

 

 

With each page table entry a valid–invalid bit is associated
(v Þ in-memory, i Þ not-in-memory)

Initially valid–invalid bit is set to i on all entries

Example of a page table snapshot:

 

 

During address translation, if valid–invalid bit in page table entry

      is I Þ page fault

  

Page Table When Some Pages Are Not in Main Memory

 

 

Page Fault

 

 

1.If there is a reference to a page, first reference to that page will trap to operating system:

2.              page fault

3.Operating system looks at another table to decide:

1.Invalid reference Þ abort

2.Just not in memory

4.Get empty frame

5.Swap page into frame

6.Reset tables

7.Set validation bit = v

8.Restart the instruction that caused the page fault

 

 

Steps in Handling a Page Fault

 

 

 

Performance of Demand Paging

 

 

 

 

Performance of Demand Paging

 

 

1.Page Fault Rate 0 £ p £ 1.0

1.if p = 0 no page faults

2.if p = 1, every reference is a fault

2.Effective Access Time (EAT)

3. EAT = (1 – p) x memory access

4. + p (page fault overhead

5.            + swap page out

6.            + swap page in

7.            + restart overhead

8.                                                     )

 

 

 

Demand Paging Example

 

 

1.Memory access time = 200 nanoseconds

 

2.Average page-fault service time = 8 milliseconds

3.EAT = (1 – p) x 200 + p (8 milliseconds)

4.         = (1 – p  x 200 + p x 8,000,000

5.              = 200 + p x 7,999,800

 

6.If one access out of 1,000 causes a page fault, then

7.         EAT = 8.2 microseconds.

8.      This is a slowdown by a factor of 40!!

 

 

 

Process Creation

 

 

 

1.Virtual memory allows other benefits during process creation:

2. – Copy-on-Write

3. – Memory-Mapped Files (later)

 

 

 

Copy-on-Write

 

 

1.Copy-on-Write (COW) allows both parent and child processes to initially share the same pages in memory

If either process modifies a shared page, only then is the page copied

 

2.COW allows more efficient process creation as only modified pages are copied

 

3.Free pages are allocated from a pool of zeroed-out pages

 

 

Before Process 1 Modifies Page C

 

 

After Process 1 Modifies Page C

 

 

 

 

What happens if there is no free frame?

 

 

1.Page replacement – find some page in memory, but not really in use, swap it out

1.algorithm

2.performance – want an algorithm which will result in minimum number of page faults

2.Same page may be brought into memory several times

 

 

Page Replacement

 

 

1.Prevent over-allocation of memory by modifying page-fault service routine to include page replacement

2.Use modify (dirty) bit to reduce overhead of page transfers – only modified pages are written to disk

3.Page replacement completes separation between logical memory and physical memory – large virtual memory can be provided on a smaller physical memory

 

 

 

Need For Page Replacement

 

 

Basic Page Replacement

 

 

1.Find the location of the desired page on disk

2.Find a free frame:
   -  If there is a free frame, use it
   -  If there is no free frame, use a page replacement algorithm to select a victim frame

3.Bring  the desired page into the (newly) free frame; update the page and frame tables

4.Restart the process

 

 

 

Page Replacement

 

 

 

 

 

 

  

Page Replacement Algorithms

 

 

1.Want lowest page-fault rate

 

2.Evaluate algorithm by running it on a particular string of memory references (reference string) and computing the number of page faults on that string

 

3.In all our examples, the reference string is

4.   

5.               1, 2, 3, 4, 1, 2, 5, 1, 2, 3, 4, 5

 

 

 

Graph of Page Faults Versus The Number of Frames

 

 

 

 

 

 

 

 

 

 

 

 

  

Optimal Page Replacement

 

 

 

 

LRU Page Replacement

 

  

1.Stack implementation – keep a stack of page numbers in a double link form:

1.Page referenced:

1.move it to the top

2.requires 6 pointers to be changed

2.No search for replacement

  

Second-Chance (clock) Page-Replacement Algorithm

 

 

 

Counting Algorithms

 

 

1.Keep a counter of the number of references that have been made to each page

2.LFU Algorithm:  replaces page with smallest count

3.MFU Algorithm: based on the argument that the page with the smallest count was probably just brought in and has yet to be used

 

 

 

Allocation of Frames

 

 

1.Each process needs minimum number of pages

2.Example:  IBM 370 – 6 pages to handle SS MOVE instruction:

1.instruction is 6 bytes, might span 2 pages

2.2 pages to handle from

3.2 pages to handle to

3.Two major allocation schemes

1.fixed allocation

2.priority allocation

 

 

Fixed Allocation

 

 

Priority Allocation

 

 

 

1.Use a proportional allocation scheme using priorities rather than size

2.If process Pi generates a page fault,

1.select for replacement one of its frames

2.select for replacement a frame from a process with lower priority number

 

 

 

Global vs. Local Allocation

 

 

1.Global replacement – process selects a replacement frame from the set of all frames; one process can take a frame from another

2.Local replacement – each process selects from only its own set of allocated frames

 

 

 

Thrashing

 

 

1.If a process does not have “enough” pages, the page-fault rate is very high.  This leads to:

1.low CPU utilization

2.operating system thinks that it needs to increase the degree of multiprogramming

3.another process added to the system

2.Thrashing º a process is busy swapping pages in and out

 

 

Demand Paging and Thrashing

 

 

1.Why does demand paging work?
Locality model

1.Process migrates from one locality to another

2.Localities may overlap

 

2.Why does thrashing occur?
S size of locality > total memory size

 

 

Memory-Mapped Files

 

 

1.Memory-mapped file I/O allows file I/O to be treated as routine memory access by mapping a disk block to a page in memory

 

2.A file is initially read using demand paging. A page-sized portion of the file is read from the file system into a physical page. Subsequent reads/writes to/from the file are treated as ordinary memory accesses.

 

3.Simplifies file access by treating file I/O through memory rather than read() write() system calls

 

4.Also allows several processes to map the same file allowing the pages in memory to be shared

 

 

 

Memory Mapped Files

 

 

 

 

Memory-Mapped Shared Memory in Windows

 

 

 

 

Allocating Kernel Memory

 

 

1.Treated differently from user memory

2.Often allocated from a free-memory pool

1.Kernel requests memory for structures of varying sizes

2.Some kernel memory needs to be contiguous

 

 

 

 

..meher

1. 1.     The Deadlock Problem
2. 2.     System Model
3. 3.     Deadlock Characterization
4. 4.     Methods for Handling Deadlocks
5. 5.     Deadlock Prevention
6. 6.     Deadlock Avoidance
7. 7.     Deadlock Detection
8. 8.     Recovery from Deadlock

1. 1.     To develop a description of deadlocks, which prevent sets of concurrent processes from completing their tasks
2. 2.     To present a number of different methods for preventing or avoiding deadlocks in a computer system.

1. 1.     Traffic only in one direction.
2. 2.     Each section of a bridge can be viewed as a resource.
3. 3.     If a deadlock occurs, it can be resolved if one car backs up (preempt resources and rollback).
4. 4.     Several cars may have to be backed up if a deadlock occurs.
5. 5.     Starvation is possible.

System Model

1. Resource types R1, R2, . . ., Rm
   1. 1.     CPU cycles, memory space, I/O devices
2. Each resource type Ri has Wi instances.
3. Each process utilizes a resource as follows:
   1. request
   2. use
   3. release

Deadlock Characterization

1. Mutual exclusion:  only one process at a time can use a resource.
2. Hold and wait:  a process holding at least one resource is waiting to acquire additional resources held by other processes.
3. No preemption:  a resource can be released only voluntarily by the process holding it, after that process has completed its task.
4. Circular wait:  there exists a set {P0, P1, …, P0} of waiting processes such that P0 is waiting for a resource that is held by P1, P1 is waiting for a resource that is held by
5.           P2, …, Pn–1 is waiting for a resource that is held by
   Pn, and P0 is waiting for a resource that is held by P0.

Resource-Allocation Graph

A set of vertices V and a set of edges E.

1. V is partitioned into two types:
   1. P = {P1, P2, …, Pn}, the set consisting of all the processes in the system.
   2. R = {R1, R2, …, Rm}, the set consisting of all resource types in the system.
2. 2.     request edge – directed edge P1 ® Rj
3. assignment edge – directed edge Rj ® Pi

Example of a Resource Allocation Graph

Resource Allocation Graph With A Deadlock

Graph With A Cycle But No Deadlock

Basic Facts

1. If graph contains no cycles Þ no deadlock.
2. If graph contains a cycle Þ
   1. if only one instance per resource type, then deadlock.
   2. if several instances per resource type, possibility of deadlock.

Methods for Handling Deadlocks

1. Ensure that the system will never enter a deadlock state.
2. Allow the system to enter a deadlock state and then recover.
3. Ignore the problem and pretend that deadlocks never occur in the system; used by most operating systems, including UNIX.

Deadlock Prevention

Restrain the ways request can be made.

1. Mutual Exclusion – not required for sharable resources; must hold for nonsharable resources.
2. Hold and Wait– must guarantee that whenever a process requests a resource, it does not hold any other resources.
   1. Require process to request and be allocated all its resources before it begins execution, or allow process to request resources only when the process has none.
   2. Low resource utilization; starvation possible.

1. No Preemption–
   1. If a process that is holding some resources requests another resource that cannot be immediately allocated to it, then all resources currently being held are released.
   2. Preempted resources are added to the list of resources for which the process is waiting.
   3. Process will be restarted only when it can regain its old resources, as well as the new ones that it is requesting.
2. Circular Wait – impose a total ordering of all resource types, and require that each process requests resources in an increasing order of enumeration.

Deadlock Avoidance

Requires that the system has some additional a priori information
available.

1. Simplest and most useful model requires that each process declare the maximum number of resources of each type that it may need.
2. The deadlock-avoidance algorithm dynamically examines the resource-allocation state to ensure that there can never be a circular-wait condition.
3. Resource-allocation state is defined by the number of available and allocated resources, and the maximum demands of the processes.

Safe State

1. When a process requests an available resource, system must decide if immediate allocation leaves the system in a safe state.
2. System is in safe state if there exists a sequence <P1, P2, …, Pn> of ALL the  processes  is the systems such that  for each Pi, the resources that Pi can still request can be satisfied by currently available resources + resources held by all the Pj, with j < i.
3. That is:
   1. If Pi resource needs are not immediately available, then Pi can wait until all Pj have finished.
   2. When Pj is finished, Pi can obtain needed resources, execute, return allocated resources, and terminate.
   3. When Pi terminates, Pi +1 can obtain its needed resources, and so on.

Basic Facts

1. If a system is in safe state Þ no deadlocks.
2. If a system is in unsafe state Þ possibility of deadlock.
3. Avoidance Þ ensure that a system will never enter an unsafe state.

Safe, Unsafe , Deadlock State

Avoidance algorithms

1. Single instance of a resource type.  Use a resource-allocation graph

1. Multiple instances of a resource type.  Use the banker’s algorithm

Resource-Allocation Graph Scheme

1. Claim edge Pi ® Rj indicated that process Pj may request resource Rj; represented by a dashed line.
2. Claim edge converts to request edge when a process requests a resource.
3. Request edge converted to an assignment edge when the  resource is allocated to the process.

1. When a resource is released by a process, assignment edge reconverts to a claim edge.
2. Resources must be claimed a priori in the system.

Resource-Allocation Graph

Resource-Allocation Graph Algorithm

1. 1.     Suppose that process Pi requests a resource Rj

1. The request can be granted only if converting the request edge to an assignment edge does not result in the formation of a cycle in the resource allocation graph

Banker’s Algorithm

1. Multiple instances.
2. Each process must a priori claim maximum use.
3. When a process requests a resource it may have to wait. 
4. When a process gets all its resources it must return them in a finite amount of time.

Data Structures for the Banker’s Algorithm

1. Available:  Vector of length m. If available [j] = k, there are k instances of resource type Rj  available.
2. Max: n x m matrix.  If Max [i,j] = k, then process Pi may request at most k instances of resource type Rj.
3. Allocation:  n x m matrix.  If Allocation[i,j] = k then Pi is currently allocated k instances of Rj.
4. Need:  n x m matrix. If Need[i,j] = k, then Pi may need k more instances of Rjto complete its task.
   1. Need [i,j] = Max[i,j] – Allocation [i,j].

Safety Algorithm

1. 1.       Let Work and Finish be vectors of length m and n, respectively.  Initialize:
   1. Work = Available
   2. Finish [i] = false for i = 0, 1, …, n- 1.
2. 2.       Find and i such that both:
   1. (a) Finish [i] = false
   2. (b) Needi £ Work
   3. If no such i exists, go to step 4.
3. 3.       Work = Work + Allocationi
   Finish[i] = true
   go to step 2.
4. 4.       If Finish [i] == true for all i, then the system is in a safe state.

Resource-Request Algorithm for Process Pi

1.       Request = request vector for process Pi.  If Requesti [j] = k then process Pi wants k instances of resource type Rj.
   1. 1.       If Requesti £ Needi go to step 2.  Otherwise, raise error condition, since process has exceeded its maximum claim.
   2. 2.       If Requesti £ Available, go to step 3.  Otherwise Pi  must wait, since resources are not available.
   3. 3.       Pretend to allocate requested resources to Piby modifying the state as follows:
      1.                    Available = Available  – Request;
      2.                    Allocationi = Allocationi + Requesti;
      3.                    Needi = Needi – Requesti;
      4. If safe Þ the resources are allocated to Pi.
      5. If unsafe Þ Pi must wait, and the old resource-allocation state is restored

Example of Banker’s Algorithm

1. 5 processes P0  through P4;
2.       3 resource types:
3.               A (10 instances),  B (5instances), and C (7 instances).
4. Snapshot at time T0:
5.                              Allocation   Max   Available
6.                              A B C          A B C          A B C
7.                    P0     0 1 0  7 5 3 3 3 2
8.                     P1    2 0 0 3 2 2 
9.                     P2    3 0 2 9 0 2
10.                     P3    2 1 1 2 2 2
11.                     P4    0 0 2  4 3 3                      

1. The content of the matrix Need is defined to be Max – Allocation.
2.                              Need
3.                              A B C
4.                     P0    7 4 3
5.                     P1    1 2 2
6.                     P2    6 0 0
7.                     P3    0 1 1
8.                     P4    4 3 1
9. The system is in a safe state since the sequence < P1, P3, P4, P2, P0> satisfies safety criteria.

Example:  P1 Request (1,0,2)

1. Check that Request £ Available (that is, (1,0,2) £ (3,3,2) Þ true.
2.                              Allocation   Need  Available
3.                              A B C          A B C          A B C
4.                    P0     0 1 0 7 4 3 2 3 0
5.                    P1     3 0 2  0 2 0  
6.                    P2     3 0 1 6 0 0
7.                    P3     2 1 1 0 1 1
8.                    P4     0 0 2 4 3 1
9. Executing safety algorithm shows that sequence < P1, P3, P4, P0, P2> satisfies safety requirement.
10. Can request for (3,3,0) by P4 be granted?
11. Can request for (0,2,0) by P0 be granted?

Deadlock Detection

1. Allow system to enter deadlock state
2. Detection algorithm
3. Recovery scheme

Single Instance of Each Resource Type

1. Maintain wait-forgraph
   1. Nodes are processes.
   2. Pi ® Pj   if Pi is waiting for Pj.
2. Periodically invoke an algorithm that searches for a cycle in the graph. If there is a cycle, there exists a deadlock.
3. An algorithm to detect a cycle in a graph requires an order of n2 operations, where n is the number of vertices in the graph.

Resource-Allocation Graph and Wait-for Graph

The Deadlock Problem

1. 1.     A set of blocked processes each holding a resource and waiting to acquire a resource held by another process in the set.
2. 2.     Example
   1. 1.     System has 2 disk drives.
   2. 2.     P1 and P2 each hold one disk drive and each needs another one.
3. 3.     Example
   1. 1.     semaphores A and B, initialized to 1
      1. 1.         P0              P1
      2. 2.     wait (A);              wait(B)
      3. 3.     wait (B);               wait(A)

Resource-Allocation Graph    Corresponding wait-for graph

Several Instances of a Resource Type

1. Available:  A vector of length m indicates the number of available resources of each type.
2. Allocation:  An n x m matrix defines the number of resources of each type currently allocated to each process.
3. Request:  An n x m matrix indicates the current request  of each process.  If Request [ij] = k, then process Pi is requesting k more instances of resource type. Rj.