Monday, September 6, 2010

Free Backup/Synch/Storage

Cloud comes to rescue!  Upload/Download any files/folders with just a drag&drop in some sites! It is 100% Free!

Windows Live SkyDrive : http://skydrive.live.com/
Windows
Windows Live SkyDrive gives you 25 GB of free online storage. SkyDrive makes it easy to access your documents, photos, and videos anywhere—and share them. The only limit: You must keep to no more than 50MB per file.

CloudDrive  http://www.driveoncloud.com/
Windows
You get 5GB of free online storage with Adobe Acrobat Online Storage Service. Uploading and downloading from such a service isn't always easy though, especially if you want to grab a whole folder of data. CloudDrive, a free client for Adobe Acrobat Online Storage Service, lets you do exactly that via drag and drop.
 

Dropbox  http://www.dropbox.com/
Windows | Mac | Linux | Web | Mobile
Our Editors' Choice for cloud-based automatic file sync, Dropbox is the epitome of a set-it-up and let-it-go service. Install it on any desktop or laptop computer and all the files in your Dropbox folder appear on all systems (or on a friend's if you share a folder). It's free for up to 2GB of data synched, and that's data you can access on your iPhone or via the Web, too.

Sunday, April 11, 2010

How to improve application web response using open source

Cassandra is a highly scalable, eventually consistent, distributed, structured key-value store. Cassandra provides a ColumnFamily-based data model richer than typical key/value systems.

Cassandraa is distributed storage system for managing structured data that is designed to scale to a very large size across many commodity servers, with no single point of failure. Reliability at massive scale is a very big challenge. At this scale, small and large components fail continuously. Cassandra manages the persistent state in the face of these failures drives the reliability and scalability of the software systems relying on this service.

Cassandra values Availability and Partitioning tolerance (AP). Tradeoffs between consistency and latency are tunable in Cassandra. You can get strong consistency with Cassandra (with an increased latency). But, you can't get row locking: that is a definite win for HBase.


Cassandra vs MySQL with 50GB of data
MySQL
Cassandra

~300ms write
~0.12ms write

~350ms read
~15ms read


Eventually Consistent
Consistency describes how and whether a system is left in a consistent state after an operation. In distributed data systems like Cassandra, this usually means that once a writer has written, all readers will see that write.
On the contrary to the strong consistency used in most relational databases (ACID for Atomicity Consistency Isolation Durability) Cassandra is at the other end of the spectrum (BASE for Basically Available Soft-state Eventual consistency). Cassandra weak consistency comes in the form of eventual consistency which means the database eventually reaches a consistent state. As the data is replicated, the latest version of something is sitting on some node in the cluster, but older versions are still out there on other nodes, but eventually all nodes will see the latest version.
More specifically: R=read replica count W=write replica count N=replication factor Q=QUORUM (Q = N / 2 + 1)


  • If W + R > N, you will have consistency
  • W=1, R=N
  • W=N, R=1
  • W=Q, R=Q where Q = N / 2 + 1
Cassandra provides consistency when R + W > N (read replica count + write replica count > replication factor).
You get consistency if R + W > N, where R is the number of records to read, W is the number of records to write, and N is the replication factor. A ConsistencyLevel of ONE means R or W is 1. A ConsistencyLevel  of QUORUM means R or W is ceiling((N+1)/2). A ConsistencyLevel  of ALL means R or W is N. So if you want to write with a ConsistencyLevel  of ONE and then get the same data when you read, you need to read with ConsistencyLevel  ALL. 

Limitations:

  • Cassandra's compaction code currently deserializes an entire row (per columnfamily) at a time. So all the data from a given columnfamily/key pair must fit in memory. 
  • Cassandra has two levels of indexes: key and column. But in super columnfamilies there is a third level of subcolumns; these are not indexed, and any request for a subcolumn deserializes _all_ the subcolumns in that supercolumn. So you want to avoid a data model that requires large numbers of subcolumns.
  • The byte size of a value can't be more than 2^31-1. 

High scalability Memcached

Memcached: Free & open source, high-performance, distributed memory object caching system, generic in nature, but intended for use in speeding up dynamic web applications by alleviating database load.
Memcached is an in-memory key-value store for small chunks of arbitrary data (strings, objects) from results of database calls, API calls, or page rendering.

Where we can use memchaced in application?
If you have a high-traffic site that is dynamically generated with a high database load that contains mostly read threads then memcached can help lighten the load on your database. If your DB load is low but CPU usage very high, you could cache computed objects and renderred templates. You can reduce writes related to session handling, temporarily stash data, cache small but frequently accessed files, cache results from web "services" or RSS feeds...

What's the limitation of memcached?
  • Objects larger than 1MB. Memcached is not for large media and streaming huge blobs.
  • The maximum size of a key is 250 characters  The maximum size of a value you can store in memcached is 1 megabyte. If your data is larger, consider clientside compression or splitting the value up into multiple keys.
  • Anyone can just telnet to any memcached server. If you're on a shared system.  Do not run in DMZ zone.
  • No persistence
  • Memcached doesn't support fail over. Remove dead nodes from your list. Be very careful! With default clients adding or removing servers will invalidate all of your cache! Since the list of servers to hash against has changed, most of your keys will likely hash to different servers. It's like restarting all of your nodes at the same time

Saturday, January 30, 2010

Open source resources of Cloud computing

Open-source software has been on the rise at many businesses during the extended economic downturn, and one of the areas where it is starting to offer companies a lot of flexibility and cost savings is in cloud computing. Cloud deployments can save money, free businesses from vendor lock-ins that could really sting over time, and offer flexible ways to combine public and private applications. The following are 11 top open-source cloud applications, services, educational resources, support options, general items of interest, and more.
Eucalyptus. Ostatic broke the news about UC Santa Barbara’s open-source cloud project last year. Released as an open-source (under a FreeBSD-style license) infrastructure for cloud computing on clusters that duplicates the functionality of Amazon’s EC2, Eucalyptus directly uses the Amazon command-line tools. Startup Eucalyptus Systems was launched this year with venture funding, and the staff includes original architects from the Eucalyptus project. The company recently released its first major update to the software framework, which is also powering the cloud computing features in the new version of Ubuntu Linux.
Red Hat’s Cloud. Linux-focused open-source player Red Hat has been rapidly expanding its focus on cloud computing. At the end of July, Red Hat held its Open Source Cloud Computing Forum, which included a large number of presentations from movers and shakers focused on open-source cloud initiatives.

Traffic Server. Yahoo this week moved its open-source cloud computing initiatives up a notch with the donation of its Traffic Server product to the Apache Software Foundation. Traffic Server is used in-house at Yahoo to manage its own traffic, and it enables session management, authentication, configuration management, load balancing, and routing for entire cloud computing software stacks. Acting as an overlay to raw cloud computing services, Traffic Server allows IT administrators to allocate resources, including handling thousands of virtualized services concurrently.
Cloudera. The open-source Hadoop software framework is increasingly used in cloud computing deployments due to its flexibility with cluster-based, data-intensive queries and other tasks. It’s overseen by the Apache Software Foundation, and Yahoo has its own time-tested Hadoop distribution. Cloudera is a promising startup focused on providing commercial support for Hadoop. You can read much more about Cloudera here.
Puppet. Virtual servers are on the rise in cloud computing deployments, and Reductive Labs’ open-source software, built upon the legacy of the Cfengine system, is hugely respected by many system administrators for managing them. You can use it to manage large numbers of systems or virtual machines through automated routines, without having to do a lot of complex scripting.
Enomaly. The company’s Elastic Computing Platform (ECP) has its roots in widely used Enomalism open-source provisioning and management software, designed to take much of the complexity out of starting a cloud infrastructure. ECP is a programmable virtual cloud computing infrastructure for small, medium and large businesses, and you can read much more about it here.
Joyent. In January of this year, Joyent purchased Reasonably Smart, a fledgling open-source cloud startup based on JavaScript and Git. Joyent’s cloud hosting infrastructure and cloud management software incorporate many open-source tools for public and private clouds.  The company can also help you optimize a speedy implementation of the open-source MySQL database for cloud use.
Zoho. Many people use Zoho’s huge suite of free, online applications, which is competitive with Google Docs. What lots of folks don’t realize, though, is that Zoho’s core is completely open source — a shining example of how SaaS solutions can work in harmony with open source. You can find many details on how Zoho deploys open-source tools in this interview.
Globus Nimbus. This open-source toolkit allows businesses to turn clusters into Infrastructure-as-a-Service (IaaS) clouds. The Amazon EC2 interface is carried over, but is not the only interface you can choose.
Reservoir. This is the main European research initiative on virtualized infrastructures and cloud computing. It’s a far-reaching project targeted to develop open-source technology for cloud computing, and help businesses avoid vendor lock-in.
OpenNebula. The OpenNebula VM Manager is a core component of Reservoir. It’s an open-source answer to the many virtual machine management offerings from proprietary players, and interfaces easily with cloud infrastructure tools and services. “OpenNebula is an open-source virtual infrastructure engine that enables the dynamic deployment and re-placement of virtual machines on a pool of physical resources,” according to project leads.
It’s good to see open-source tools and resources competing in the cloud computing space. The end result should be more flexibility for organizations that want to customize their approaches. Open-source cloud offerings also have the potential to keep pricing for all competitive services on a level playing field.

Limitations of ESX 4.0 open source

Infrastructure limitations of ESX 4.0 VMware

Some limitations in ESX Server 4 may constrain the design of data centers:
  • Guest system maximum RAM: 255 GB
  • Host system maximum RAM: 1TB
  • Number of hosts in an HA cluster: 64
  • Number of hosts in a DRS cluster: 64
  • Maximum number of processors per virtual machine: 8
  • Maximum number of processors per host: 64
  • Maximum number of cores per processor: 6

Performance limitations

In terms of performance, virtualization imposes a cost in the additional work the CPU has to perform to virtualize the underlying hardware. Instructions that perform this extra work, and other activities that require virtualization, tend to lie in operating system calls. In an unmodified operating system, OS calls introduce the greatest portion of virtualization overhead.

Sources for free virtual machine software


Here are a few sources of free virtual machine software. Some of these are sources of true virtual appliances and some are just virtual machines with an OS pre-installed.
  • Xen-get - Free virtual machine images for Xen. http://xen-get.org/. Most of the major free Linux distributions are available pre-installed. These are actually just file system images, but when used with Xen the function is the same as a full virtual machine image.
  • Jailtime - More Free virtual machine images for Xen. http://jailtime.org. This site is pretty similar to Xen-get and like Xen-get these are file system images.
  • OSZoo - Free virtual machine images for Qemu. http://www.oszoo.org/wiki/index.php/Main_Page. Again the typical set of Linux and other open source OS VM images. The interesting thing about this site is that if you use Q for OS X you can download images directly from the application. It’s also possible to convert these images for use with Parallels and VMWare and to do the same with Xen when running on hardware that supports the hvmloader [1] to boot an unmodified operating system.
  • VMWare Appliance Directory - Lot’s virtual appliances and virtual machine images for VMWare. http://www.vmware.com/vmtn/appliances/. This is the big list of virtual appliances for VMWare. There’s somewhere around 170 appliances online now, some of which are really nice. With a little work most of these can also be converted to run under Parallels or Xen(on an hvm enabled platform).
  • rPath rBuilder Online - An online tool for building open source appliances. http://www.rpath.com/rbuilder/. Some of these are available through the VMWare directory, but there are other projects there that aren’t currently listed. From the rBuilder site you can also sometimes find images that can be used directly in Parallels and Xen.
  • Virtualappliances.net - This company builds some of the best examples of virtual appliances currently available. They’re small and very focused. http://virtualappliances.net/. They currently have appliances for MySQL, PostgreSQL, Tomcat and a simple web server appliance. All appliances are available for VMWare and Xen (no hvm support required).

Friday, January 1, 2010

One-stop to verify your Application in various browser

Browsershots makes screen shots of your web design in different browsers. It is a free open-source online service. When you submit your web address, it will be added to the job queue. A number of distributed computers will open your website in their browser. Then they will make screen shots and upload them to the central server.  You will get a report of all browser screen shot images of your web site. http://browsershots.org/