You can get support 24\7 here absolutely free.
Use the contacts box to the right of the blog to find email, skype or icq for asking question and retrieving immediate answers.
Feel free to order developments, investigations and so forth concerned with Kapow Robosuite Technology and any kind of programming.
Thursday, July 24, 2008
Kapow Robosuite Fuctionality
What can be done with Kapow Robosuite Techology:
1. Load any web content and content from files.
2. Change content while loading it, - add, edit or delete. Strong support of regular expressions.
3. Save original or edited content into database, file or other webpage, webservice
4. Call any webservice to get data
5. Schedule robot runs to make data extraction process automated. Data is kept up-to-date.
6. Deploy your own webservice using kapow robosuite web 2.0. edition
7. Automaticly fill your portal with extracted data.
8. Use C# and Java API to manipulate robot runs, extracted data processing and deployment. Unlimited functionality.
1. Load any web content and content from files.
2. Change content while loading it, - add, edit or delete. Strong support of regular expressions.
3. Save original or edited content into database, file or other webpage, webservice
4. Call any webservice to get data
5. Schedule robot runs to make data extraction process automated. Data is kept up-to-date.
6. Deploy your own webservice using kapow robosuite web 2.0. edition
7. Automaticly fill your portal with extracted data.
8. Use C# and Java API to manipulate robot runs, extracted data processing and deployment. Unlimited functionality.
Kapow Deployment Schema
This article will explain what is Kapow Robosuite like, when it is deployed and working.
First of all the launch of robots is done in 1 of the following applications within Kapow Robosuite installation.
1. Kapow Roboserver
It is a process on linux or windows server, which working on the port, you mentioned while installation. Generally it is 50000 by default for Data Collection Edition.
2. Kapow RoboRunner
Kapow Roborunner is not represented as a process. It is launched when you want to launch robots from command line or from crontab. It is ddifferent from roboserver, because we cannot pass parameters to robots when using it. From the other side - the robots can be launched using crontab or windows scheduler, that is easing the process.
After we decide which of them to use - we create robots to make data migration for example.
Robots are made in Robomaker and are generally stored in project/library/ path within installation of robosuite. Robots need models to store some temporary calculations, data and so forth. Models are stored where the robots are, but they are created by ModelMaker application.
Launching Robots:
1. RoboClient - the most easy way. Just set up the ip address of the roboserver, path to robot and click launch.
2. RoboManager - also easy to use, but it is more powerful than roboclient, as it manages robot groups, outputs, errors, etc...
3. API - C# or java APIs for launching. You can make 3-chained calls for php or other languages.
4. Command line - in crontab, or just in console. It is only for Roborunner.
First of all the launch of robots is done in 1 of the following applications within Kapow Robosuite installation.
1. Kapow Roboserver
It is a process on linux or windows server, which working on the port, you mentioned while installation. Generally it is 50000 by default for Data Collection Edition.
2. Kapow RoboRunner
Kapow Roborunner is not represented as a process. It is launched when you want to launch robots from command line or from crontab. It is ddifferent from roboserver, because we cannot pass parameters to robots when using it. From the other side - the robots can be launched using crontab or windows scheduler, that is easing the process.
After we decide which of them to use - we create robots to make data migration for example.
Robots are made in Robomaker and are generally stored in project/library/ path within installation of robosuite. Robots need models to store some temporary calculations, data and so forth. Models are stored where the robots are, but they are created by ModelMaker application.
Launching Robots:
1. RoboClient - the most easy way. Just set up the ip address of the roboserver, path to robot and click launch.
2. RoboManager - also easy to use, but it is more powerful than roboclient, as it manages robot groups, outputs, errors, etc...
3. API - C# or java APIs for launching. You can make 3-chained calls for php or other languages.
4. Command line - in crontab, or just in console. It is only for Roborunner.
Monday, July 21, 2008
Kapow Robosuite is a Dream for Implementing Business Ideas
In this post i would like to describe the possible ways of applying Kapow Robosuite Technology for business ideas(money is the aim).
Here we can follow these directions:
1. Aggregate data, which already exists and just needs to be compiled in 1 place
For example you own a wonderful IT blog and want to keep it up-to-date. You hardly can monitor the rapid developing IT market for new solutions, technologies and innovations. So the use of kapow roboserver can help you to do several things simultaneously:
1. Collect the most recent IT news from theme websites.
2. Collect most recent product releases from website owners like blackberry, apply, microsoft, sourcefourge etc..
3. Collect user opinions on point 1 and 2 and post it to your blog
4. Collect theme-friendly content and place it to your blog as well(For example URL of shops where to buy different gadgets, shipping costs etc..).
2. Analyze data, which already exists somewhere, but needs to be interpreted in some way to make a knowledge base
Lets take the prevous example. We are going to expand it.
1. We make a robot in robomaker for each shop, where you can buy mobile phones for example
2. The robot collects all the characteristics for each mobile device and puts it into DB
3. You make a web application for comparing products
4. You see the result like this one: http://tele.clabs.eu/ (Though it is in russian language, i hope it is understandable)
3. Imitation of user regular actions
You have 5 email addresses, 12 blogs and 15 social networks accounts?
Have this info and actions in 1 place!!!
It is just a matter of time to create needed amount of Kapow Robosuite robots and web interface for handling this all. It is worth it, anyway.
To be continued...
Here we can follow these directions:
1. Aggregate data, which already exists and just needs to be compiled in 1 place
For example you own a wonderful IT blog and want to keep it up-to-date. You hardly can monitor the rapid developing IT market for new solutions, technologies and innovations. So the use of kapow roboserver can help you to do several things simultaneously:
1. Collect the most recent IT news from theme websites.
2. Collect most recent product releases from website owners like blackberry, apply, microsoft, sourcefourge etc..
3. Collect user opinions on point 1 and 2 and post it to your blog
4. Collect theme-friendly content and place it to your blog as well(For example URL of shops where to buy different gadgets, shipping costs etc..).
2. Analyze data, which already exists somewhere, but needs to be interpreted in some way to make a knowledge base
Lets take the prevous example. We are going to expand it.
1. We make a robot in robomaker for each shop, where you can buy mobile phones for example
2. The robot collects all the characteristics for each mobile device and puts it into DB
3. You make a web application for comparing products
4. You see the result like this one: http://tele.clabs.eu/ (Though it is in russian language, i hope it is understandable)
3. Imitation of user regular actions
You have 5 email addresses, 12 blogs and 15 social networks accounts?
Have this info and actions in 1 place!!!
It is just a matter of time to create needed amount of Kapow Robosuite robots and web interface for handling this all. It is worth it, anyway.
To be continued...
Using Kapow Robosuite and avoiding ban on websites
Some websites do not allow to parse information or make some actions in intensive way.
The examples of such website could be:
1. Air tickets company portal - like germanwings, lufthansa, windjet.
2. Google
3. Digg.com, Reddit.com
There are two ways of stopping automated access to their data:
1. Captchas
2. Ban by IP after reaching some limit of requests.
Ways of avoiding ban:
1. Captchas - Kapow Robosuite Technology allows creating clipping robots, which do all needed background job and when a user action is needed - they output the subresult into portlet or servlet, wait for your action. After action is commited the roboserver interprets the result and makes further actions. So is with captchas as well.
For example we want to automate registration process on google.com
Our steps:
1. We create a clipping robot, which fills in the for fields like name, nick, password, email(all could be taken from DB or from another website on the fly). The robot ends with END CLIP step, which will output only captcha as a result.
2. We deploy the robot on roboserver, launch it hrough browser and see the captcha and submit button only(all filled in fields are in background and are not essential for viewing. Though it is an option).
3. We enter captcha's word, click enter. This causes the robot to continue from the step after we filled in the other fields. It submits form and account is created!!! So simple. Some websites need email confirmation - no problem. We just go with the robot to the given email and click the activation link. We even can wait(It is a kapow robomaker step with param of wait interval) a bit to avoid the situation when email is delivered with latency.
4. We save registration info wherever we need: DB, text file, excell or more.
Another example:
Google doesnt allow you to make thousands searches a day. It bans you by ip address.
Kapow Robosuite based Solution:
1. You purchase proxy lists subscriptions from 3rd party. It is better to have an hourly refreshed list then 10000 proxies at once. We need living proxies, not dead ones.
2. You make a robot which grabs these proxy lists from the place, where the are available. It saves the in needed format to configuration/proxyserver.properties.
The format is:
AnyUniqueName.proxyServerHost=122.213.250.10
AnyUniqueName.proxyServerPort=80
AnyUniqueName.proxyServerUse=true
3. Then you make a needed robot for querying google and add CHANGE PROXY step at the beggining of each request(or each 5-th request for example).
That is it. Now you can make requests as often as you need.
The examples of such website could be:
1. Air tickets company portal - like germanwings, lufthansa, windjet.
2. Google
3. Digg.com, Reddit.com
There are two ways of stopping automated access to their data:
1. Captchas
2. Ban by IP after reaching some limit of requests.
Ways of avoiding ban:
1. Captchas - Kapow Robosuite Technology allows creating clipping robots, which do all needed background job and when a user action is needed - they output the subresult into portlet or servlet, wait for your action. After action is commited the roboserver interprets the result and makes further actions. So is with captchas as well.
For example we want to automate registration process on google.com
Our steps:
1. We create a clipping robot, which fills in the for fields like name, nick, password, email(all could be taken from DB or from another website on the fly). The robot ends with END CLIP step, which will output only captcha as a result.
2. We deploy the robot on roboserver, launch it hrough browser and see the captcha and submit button only(all filled in fields are in background and are not essential for viewing. Though it is an option).
3. We enter captcha's word, click enter. This causes the robot to continue from the step after we filled in the other fields. It submits form and account is created!!! So simple. Some websites need email confirmation - no problem. We just go with the robot to the given email and click the activation link. We even can wait(It is a kapow robomaker step with param of wait interval) a bit to avoid the situation when email is delivered with latency.
4. We save registration info wherever we need: DB, text file, excell or more.
Another example:
Google doesnt allow you to make thousands searches a day. It bans you by ip address.
Kapow Robosuite based Solution:
1. You purchase proxy lists subscriptions from 3rd party. It is better to have an hourly refreshed list then 10000 proxies at once. We need living proxies, not dead ones.
2. You make a robot which grabs these proxy lists from the place, where the are available. It saves the in needed format to configuration/proxyserver.properties.
The format is:
AnyUniqueName.proxyServerHost=122.213.250.10
AnyUniqueName.proxyServerPort=80
AnyUniqueName.proxyServerUse=true
3. Then you make a needed robot for querying google and add CHANGE PROXY step at the beggining of each request(or each 5-th request for example).
That is it. Now you can make requests as often as you need.
Kapow Robosuite Technology Overview
Kapow Robosuite is a powerful instrument, which is designed to solve the following tasks:
1. Parsing, Grabbing, Harvesting, Data Migration
Using kapow products you can easily convert any web content into your databases, excel files, word documents. And this is not the limit. By using 3rd part applications you can convert harvested data into any specific format you can imagine(PDF, Images, etc...).
2. Imitation of user behaiviour in Internet
It means that you are able to clone any of your regular actions and save time and money. For example you need to receive notifications when you are running out of funds on some of websites, which service you are using. No need to check it daily. Just make a robot, launched automaticly daily, which will send you an email or SMS with warning.
3. Content aggregation, portals enhacements
You daily visit websites about weather, stock rates, politic news on google and more? Join them into one portal by using kapow robosuite technology. It is fast, it is reliable, it is useful.
4. Making your own dynamic Mashups
Mashup is the process of leveraging the informational and functional resources
of the Web in an application of your choice. So using kapow robosuite web 2.0. edition will allow you to represent any data as Widgets, Web Services, Traditional Web Applications.
1. Parsing, Grabbing, Harvesting, Data Migration
Using kapow products you can easily convert any web content into your databases, excel files, word documents. And this is not the limit. By using 3rd part applications you can convert harvested data into any specific format you can imagine(PDF, Images, etc...).
2. Imitation of user behaiviour in Internet
It means that you are able to clone any of your regular actions and save time and money. For example you need to receive notifications when you are running out of funds on some of websites, which service you are using. No need to check it daily. Just make a robot, launched automaticly daily, which will send you an email or SMS with warning.
3. Content aggregation, portals enhacements
You daily visit websites about weather, stock rates, politic news on google and more? Join them into one portal by using kapow robosuite technology. It is fast, it is reliable, it is useful.
4. Making your own dynamic Mashups
Mashup is the process of leveraging the informational and functional resources
of the Web in an application of your choice. So using kapow robosuite web 2.0. edition will allow you to represent any data as Widgets, Web Services, Traditional Web Applications.
Subscribe to:
Posts (Atom)