MatLab Resources

SQL DB Library | XML Generator | WebBot | Input Dialog | Program Statistics | Triangular Random | Fractals Explorer ]
© CSE, 2003-2004


CSE Web


WebBot v.2.0

Internet page analysis/download utility

This function is a Java-based "web browser" that extract all links from a web-page, and display them. The resulting documents can be downloaded. Its conception can be used as a basic example for using the java.net.url Java Class, as well as PERL-originating regular expressions.

Content and usage

webbot(URL)URL is a string indicating the base page address; the url must link to an html file. The function lists all links in the file. URL can also be a cell vector of url-strings.
Selection
webbot(URL, WHAT)displays only specific links. WHAT is a string:
 'all_links'displays all links (default).
 'page_links'displays all links to an html web page*.
 'local_links'displays all local links on the server*.
 'external_links'displays all links to external websites.
 'image_links'displays all links to an image file*.
 'image_tags'displays all image tags <img src="xxx">.
 '.xxx.yyyy.zz'displays all links to each specific .xxx files; the case is ignored ('zip' will find 'ZiP'); Example: '.zip.gz.gzip.tar.Z'.
Action
webbot(URL, WHAT, ACT)performs an action on found links. ACT is a string:
 'noaction'just display links (default)
 'download'downloads all links found.
 'cartoons'downloads all image links found on linked pages. This is usefull e.g. for cartoons websites where each cartoon (e.g. "01.gif") is on its own html page (e.g. "c01.html")*.
 'follow.x'follows links to html pages and recursively performs the same action on the resulting page. 'x' is an integer indicating the recursivity depth (0 is equivalent to 'noaction').
Return value
lks = webbot(URL, ...)returns an cell-array with links of URL{end}.


Version and download

Version 2.0 - Release 20.12.03 [
Legal Stuff]
Download the function as a .zip (6 Kb).

Version 1.0 - Release 15.10.03 [Legal Stuff]
Download the function as a .m matlab file (18 Kb).

Limitations/Prerequisites

As a standard MatLab installation comes with a Java Virtual Machine, there are no pre-requisites. This has been developed and solely tested on Release 13. There are some limitations in this early version, that may, or may not be corrected/completed:
  • Only english international characters are recongnized. A shame for a french-speaking living in a german-speaking region, é, à, ç or other ï are not any more supported than ä, ö or ü. This is usually not a problem as URLs are usually given names in standard "ANSI" characters...
  • Links are recognized in the flags 'page_links' and 'local_links' only if they are explicitely pointing to a .htm or .html url, i.e. not if they point on a folder with implicit call to index.html or index.htm. This has been partially fixed in version 2.0, but it is not sure that it will work seamlessly for lack of testing...
  • Image links (in the flag 'image_links') are recognized solely by file endings, namely the following file types: .jpg .jpeg .gif .pict .bmp .tif .tiff .ras .png (.giff)
  • The return value should be a consolidated list of all links found, not just the ones of the last url visited in the first call.

Changes from 1.0 to 2.0:

In version 1.0, the binary download was unbuffered, i.e. one byte is read at a time. It was mentioned that when I would understand how I can send a java array reference pointer to java, this can be modified (for an expected speed gain of at least factor 5).
 Note that the problem was not in using:
jA = javaArray('java.lang.Byte',n)
but in the fact that the 'read(byte[])' method of an 'inputStream' object does not accept the above defined jA as parameter.
If somebody can help, please do!
Now this has been modified, having learned that:
  • It is impossible in R13 to pass arrays by reference in Java functions.
  • Matworks provide a cool class (.getInterruptibleStreamCopier) that does the job perfectly. Indeed Mathworks even provide a built-in function to download files: urlwrite. Well, always nice to re-invent the wheel.
So now the download is buffered, and the speed gain is much higher than the expected factor 5!

Example:

Try this out with Dilbert's archives to download a full month's value of Dilbert's comics of the day!
webbot('http://www.unitedmedia.com/comics/dilbert/archive/', 'local_links', 'cartoons');
Remember these pictures are copyrighted and support the making of new comics by buying Scott Adam's books!

top of page



CSE Flag Dialog v.1.1

GUI Dialog for user input

CSE Flag Dialog makes a GUI dialog to select from options of diverse types (checkboxes, radiobuttons, text-inputs and popup lists). Some options can be linked, i.e. be mutually exclusive or be only enabled according to the value of another option.
It is an extension of CSEOptionDialog.

Features and Usage

out = CSEFlagDialog(items)
out = CSEFlagDialog(items, title)
out = CSEFlagDialog(items, title, msg)
out = CSEFlagDialog(items, title, msg, options)

items is an array structure containing the options, in the following fields:
.name is a string (the name of the option)
.values is a cell array of possible answer:
if empty, it is considered to contain "Yes" and "No", i.e. the option is a checkbox or a radio-button.
if it contains only one numeric element, it is considered a header, and not an active control.
if it contains only one text element, it is considered to be a text input field, with the .values as default value.
Otherwise, it is a popup list with the choices given in values.
.linked [optional field] is an array of index in items indicating which options are linked with the option. Linked options will be grayed out when the option is set to "No".
If .linked contains negative elements, those will be grayed out when the option is set to "Yes".
.exclusive    [optional field] is an array of index in items indicating which options are mutually exclusive with each option. When the option is set to "Yes", mutually exclusive options are set to "No". If the field does not exist, or is empty, the control if a checkbox otherwise it is a radio-button.
Both .linked and .exclusive behaviour are only implemented for "Yes"/"No" fields - but any field can be in "linked" and will be grayed out.
.default [optional field] is an integer indicating the default value for each option (0: No, 1: Yes, -1: grayed out). Note that in the case of text input field, this field is irrelevant and the default is given in the "values" field; In the case of popup lists, the default is an index in .values, or -1 (grayed out).
"Linked" and "Exclusive" are NOT evaluated in the initial layout of the dialog, hence the default must be consequent with the rules, e.g. linked fields of a "No" field must be grayed out.
.indent [optional field] allows an indentation (proportional to the value of this field) from the position of the control. May be used to graphically make "groups" as no "frames" are used.
.help [optional field] contains tooltips help texts. Can contain "\n" to make multi-line help fields.
The other (optional) inputs are:
title is the window title (string)
msg is a string appearing on the top of the dialog
options is an optional structure containing options:
.center = 0|1 (center text msg)
.bold = 'light'|'normal'|'demi'|'bold': indicates how headers (see .values) must be printed out.
[more to come in future versions]

The controls will be display in the order they appear in items.

out contains an array of structure of answers:
out(i).answer = value of the control.
values are:
  • 1 or 0 for "Yes"/"No" controls,
  • the index of the chosen item for list controls.
  • the text given for text inputs.
out is empty if cancel was chosen, negative integer if error.

Version and download

Version 1.1 - Release 07.12.03 [
Legal Stuff]

Download the function as a .m matlab file (18 Kb).
Tested under R13 and R12 - no toolbox required.

Example:



Code:
items(1).name = 'Contact:';
items(1).default = 0;
items(1).values = {'email@address'};
items(1).help = 'Enter one of:\n- your email\n
    - your phone number.';
items(2).name = 'I will be coming!';
items(2).default = 1;
items(2).linked = [3 4 5 6];
items(3).name = 'With my family';
items(3).default = 1;
items(3).exclusive = 4;
items(3).indent = 1;
items(4).name = 'Alone';
items(4).default = 0;
items(4).exclusive = 3;
items(4).indent = 1;
items(5).name = 'Transportation:';
items(5).indent = 1;
items(5).values = {1};
items(6).name = 'Coming by';
items(6).default = 1;
items(6).indent = 2;
items(6).values = {'Train'; 'Bus'; 'Foot'};
items(6).help = 'Cars are polluting.\nUse public
    transportation!';
items(7).name = 'I''ll sure give a phone call!';
items(7).default = 0;

title = 'Birthday party incsription';

msg = sprintf(['Dear friends,\nAs you all know,
  I am turning 30 next april.\nThis seems a worthy
  occasion to party a bit!\n\nWill you be able to
  attend?']);

out = CSEFlagDialog(items, title, msg)


top of page



Program_statistics v.1

Statistics about multi-file programs

This simple m-file compiles some statistics about a program.A report is printed; statistics can be returned in structure. The statistics include:
  • number of files,
  • number of functions and GUIDE-related functions,
  • number of lines, words and characters,
  • number of page to print out the whole program.

Features and Usage

program_statistics(direct)
stats = program_statistics(direct);
stats = program_statistics(direct, options);

  • direct: directory where the program resides
  • options: a structure with the following optional fields:
    • .recursive = 0|1 take also into account subfolders recursively
    • .complete = 0|1 compile extended set of statistics (GUIDE-related, comments related, ...)
    • .include cell array of text files types (.xxx) to include in the statistics
      (e.g. {'.txt'; '.c'})
The program will optionally return a stats structure containing all information present in the report:

EXAMPLE OF REPORT:
%
% Number of folders parsed: 44
% Number of code folders: 8 (22.125 files per folder)
%
% Number of files: 177
% - including 588 functions (3.322 functions per file),
% - including 25 GUI dialog callback files made with GUIDE,
% (linked with .fig files weighting 657.3438 Kb)
% - including 335 GUIDE-related functions,
% - including 253 other functions (1.6645 functions per file).
%
% Number of lines: 40600
% - distributed in 172234 words,
% - including 2774 (6.8325%) separation lines,
% - including 8606 (21.197%) lines of comments,
% - including 29220 (71.9704%) lines of code.
%
% Number of characters: 1549343
% - including 407452 (26.2984%) comment characters,
% - including 1141891 (73.7016%) code characters.
%
% To print the whole program, 1014 A4-pages would be necessary.

Version and download

Version 1.0 - Release 21.06.04 [
Legal Stuff]

Download the function as a .m matlab file (10 Kb).

Limitations/Prerequisites

None known. Tested exclusively with MatLab R13.

top of page



TriRnd v.2.0

Triangular random numbers

This simple m-file generates discrete random numbers from a triangular distribution.
The distribution is defined by:
  • a minimum and a maximum value
  • a "top" value, with the highest probability.
The distribution is defined with zero probability at minVal-1 and maxVal+1, and with highest probability at topVal. Hence every value in the range (including the maximum and minimum values) have a non-zero probability to be included, whatever topValue is.
The output is a matrix containing NxM random integers.

Version and download

Version 2.0 - Release 20.10.04 [
Legal Stuff]

Download the function as a .m matlab file (3 Kb).

Limitations/Prerequisites

This is a numeric approximation, so use with care in "serious" statistical applications! It does necessitate Matlab and the statistical toolbox.
  • Two different algorithms are implemented. One is efficient for large number of random points within a small range (maxVal-minVal), while the other is efficient for large range for reasonable number of points. For large ranges, there is a O(n2) relation with regard to the product of range * number_of_points. When this product reach about a billion, the runtime reach several minutes.
  • To inspect the resulting distribution, plot a histogram of the resulting random numbers, e.g. "hist(trirnd(1,87,100,10000,1),100)".

top of page


Legal blahblah & Conditions of use
© CSE - L.Cavin, 2003, 2004
These libraries and functions (THE PROGRAM) are provided "as is" without warranty, either expressed or implied, including, but not limited to, the implied warranties of merchantability and fitness for a particular purpose. The entire risk as to the quality and performance of the program is with the person downloading or running the program (THE USER). Should the program prove defective, the user assumes the cost of all necessary servicing, repair or correction.
In no event will the Author or the Copyright holder be liable to the user for damages, including any general, special, incidental or consequential damages arising out of the use, peruse or inability to use the program (including, but not limited to loss of data or data being rendered inaccurate or losses sustained by the user or third parties or a failure of the program to operate with any other programs), even if the Author or the Copyright holder has been advised of the possibility of such damages.
The fact of downloading or running the program implies acceptation of the present liability limitation by the user.
These libraries and functions are free to use for non-commercial purposes and can be distributed (free of charge) as long as the copyrights notices are kept intact. In particular, if the program is distributed further by the user, the user is responsible for including this legal warning and liability limitation to the distribution.
It is also encouraged to improve these functions; please send to the author any improvements - we may want to include them in this distribution under the same conditions.


Do not be afraid, I am confident that the program will work - without warranties of course ;-)
Thanks to the Free Software Foundation for inspiring this nice little legal blahblah... My favorite part is the "inability to use"... even if the "general, special, incidental or consequential" part is also quite fun!


Page ©
Cavin Softwares Engineering, 2003-2004.
[last updated: 25.10.04]