Discussion:
Write / Read cache in different devices
Roberto Alcântara
2013-04-15 13:53:04 UTC
Permalink
Hi,

What you think about to have differente devices to read and write (dirty?)
caches ?

I always have raid-1 flash drives used with flashcache because my
enviroment need to have disk protection (128GB ssd disks).

But I'm thinking about how to optimize that so if I include one more sdd
and flashcache put only read cache to this device and keep putting in
raid-1 array only write (dirty?) pages?

I know flashcache do not have this feature yet. I just want to know what
you guys think about.

regards,
- Roberto
--
You received this message because you are subscribed to the Google Groups "flashcache-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to flashcache-dev+***@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
William Smith
2013-04-15 23:00:47 UTC
Permalink
If blocks go into the write (behind?) cache on one SSD , how do they then
get onto the read cache SSD ? Because presumably blocks written are more
likely than others to be read again soon.

It's true that SSD write speed is critical because HDD reads speed is
constrained by SSD writes. If this is hitting your performance , consider
the mode where you can skip sequential reads going to the SSD read cache.

Regards

Will Smith
Post by Roberto Alcântara
Hi,
What you think about to have differente devices to read and write (dirty?)
caches ?
I always have raid-1 flash drives used with flashcache because my
enviroment need to have disk protection (128GB ssd disks).
But I'm thinking about how to optimize that so if I include one more sdd
and flashcache put only read cache to this device and keep putting in
raid-1 array only write (dirty?) pages?
I know flashcache do not have this feature yet. I just want to know what
you guys think about.
regards,
- Roberto
--
You received this message because you are subscribed to the Google Groups
"flashcache-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
For more options, visit https://groups.google.com/groups/opt_out.
--
You received this message because you are subscribed to the Google Groups "flashcache-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to flashcache-dev+***@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
Roberto Alcântara
2013-04-16 12:08:10 UTC
Permalink
Will,

"write device" don't need to be write only. I think write can be done just
like now (talking about write-back). And let them cached on ssd. But if
you read something from disks you will let this data on read-only ssd. And
if you need to remove an page from raid array, you can copy them to
read-only ssd (not sure if this is good).

So with 4x 128GB SSDs I will have 128GB SSD Raid1 used to keep dirty
write pages exclusively (and other pages if not full with dirty) plus
256GB to keep all no dirty pages. This seem more usefull instead to have
256GB raid arrays, once we can more data on cache without forget safety.
Of course read intensive workloads can be most benefited from this approach

What you think about ? I'm looking for flashcache code and thinking about
how to change code to improve my skills, but will be nice try something
usefull.







- Roberto
Post by William Smith
If blocks go into the write (behind?) cache on one SSD , how do they then
get onto the read cache SSD ? Because presumably blocks written are more
likely than others to be read again soon.
It's true that SSD write speed is critical because HDD reads speed is
constrained by SSD writes. If this is hitting your performance , consider
the mode where you can skip sequential reads going to the SSD read cache.
Regards
Will Smith
Post by Roberto Alcântara
Hi,
What you think about to have differente devices to read and write
(dirty?) caches ?
I always have raid-1 flash drives used with flashcache because my
enviroment need to have disk protection (128GB ssd disks).
But I'm thinking about how to optimize that so if I include one more sdd
and flashcache put only read cache to this device and keep putting in
raid-1 array only write (dirty?) pages?
I know flashcache do not have this feature yet. I just want to know what
you guys think about.
regards,
- Roberto
--
You received this message because you are subscribed to the Google Groups
"flashcache-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
For more options, visit https://groups.google.com/groups/opt_out.
--
You received this message because you are subscribed to the Google Groups
"flashcache-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
For more options, visit https://groups.google.com/groups/opt_out.
--
You received this message because you are subscribed to the Google Groups "flashcache-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to flashcache-dev+***@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
Roberto Alcântara
2013-04-16 12:35:57 UTC
Permalink
And of course you don't need to have all ssd disk to one or other way.
Using soft raid you can optimize read and write(dirty) cache size, once
dirty cache will use twice ssd space.


- Roberto


On Tue, Apr 16, 2013 at 9:08 AM, Roberto Alcântara
Post by Roberto Alcântara
Will,
"write device" don't need to be write only. I think write can be done just
like now (talking about write-back). And let them cached on ssd. But if
you read something from disks you will let this data on read-only ssd. And
if you need to remove an page from raid array, you can copy them to
read-only ssd (not sure if this is good).
So with 4x 128GB SSDs I will have 128GB SSD Raid1 used to keep dirty
write pages exclusively (and other pages if not full with dirty) plus
256GB to keep all no dirty pages. This seem more usefull instead to have
256GB raid arrays, once we can more data on cache without forget safety.
Of course read intensive workloads can be most benefited from this approach
What you think about ? I'm looking for flashcache code and thinking about
how to change code to improve my skills, but will be nice try something
usefull.
- Roberto
Post by William Smith
If blocks go into the write (behind?) cache on one SSD , how do they
then get onto the read cache SSD ? Because presumably blocks written are
more likely than others to be read again soon.
It's true that SSD write speed is critical because HDD reads speed is
constrained by SSD writes. If this is hitting your performance , consider
the mode where you can skip sequential reads going to the SSD read cache.
Regards
Will Smith
Post by Roberto Alcântara
Hi,
What you think about to have differente devices to read and write
(dirty?) caches ?
I always have raid-1 flash drives used with flashcache because my
enviroment need to have disk protection (128GB ssd disks).
But I'm thinking about how to optimize that so if I include one more sdd
and flashcache put only read cache to this device and keep putting in
raid-1 array only write (dirty?) pages?
I know flashcache do not have this feature yet. I just want to know what
you guys think about.
regards,
- Roberto
--
You received this message because you are subscribed to the Google
Groups "flashcache-dev" group.
To unsubscribe from this group and stop receiving emails from it, send
For more options, visit https://groups.google.com/groups/opt_out.
--
You received this message because you are subscribed to the Google Groups
"flashcache-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
For more options, visit https://groups.google.com/groups/opt_out.
--
You received this message because you are subscribed to the Google Groups "flashcache-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to flashcache-dev+***@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
Mohan Srinivasan
2013-08-13 15:36:56 UTC
Permalink
Hi

1) I created a new stable_v2.1 branch.
2) In master, post stable_v2.1, I committed all of the changes from the facebook internal flashcache repository. These include changes for better IO scaling, fine grained (per cache set) locking of flashcache, improved replacement algorithm (2 Queue) and several fixes.

I will be updating the documentation detailing these changes over the next few days.

Please let me know if you see issues.

thanks

mohan
--
You received this message because you are subscribed to the Google Groups "flashcache-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to flashcache-dev+***@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
Mohan Srinivasan
2013-08-13 15:48:43 UTC
Permalink
One more note on the changes just committed - The IO scaling changes in master change the on-ssd cache format. But the changes are compatible with the old format. The code detects when the existing format on-ssd is old and reverts to old layout seamlessly.

mohan


________________________________
From: Mohan Srinivasan <***@yahoo.com>
To: "flashcache-***@googlegroups.com" <flashcache-***@googlegroups.com>
Sent: Tuesday, August 13, 2013 8:36 AM
Subject: [flashcache-dev] New flashcache branch and lots of facebook scaling changes committed...



Hi

1) I created a new stable_v2.1 branch.
2) In master, post stable_v2.1, I committed all of the changes from the facebook internal flashcache repository. These include changes for better IO scaling, fine grained (per cache set) locking of flashcache, improved replacement algorithm (2 Queue) and several fixes.

I will be updating the documentation detailing these changes over the next few days.

Please let me know if you see issues.

thanks

mohan
--
You received this message because you are subscribed to the Google Groups "flashcache-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to flashcache-dev+***@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
--
You received this message because you are subscribed to the Google Groups "flashcache-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to flashcache-dev+***@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
Sharib Jafari
2013-08-13 16:12:50 UTC
Permalink
*Great !!!*

What about cache resizing ?
Any development in that area ?


On Tue, Aug 13, 2013 at 9:18 PM, Mohan Srinivasan <
Post by Mohan Srinivasan
One more note on the changes just committed - The IO scaling changes in
master change the on-ssd cache format. But the changes are compatible with
the old format. The code detects when the existing format on-ssd is old and
reverts to old layout seamlessly.
mohan
------------------------------
*Sent:* Tuesday, August 13, 2013 8:36 AM
*Subject:* [flashcache-dev] New flashcache branch and lots of facebook
scaling changes committed...
Hi
1) I created a new stable_v2.1 branch.
2) In master, post stable_v2.1, I committed all of the changes from the
facebook internal flashcache repository. These include changes for better
IO scaling, fine grained (per cache set) locking of flashcache, improved
replacement algorithm (2 Queue) and several fixes.
I will be updating the documentation detailing these changes over the next few days.
Please let me know if you see issues.
thanks
mohan
--
You received this message because you are subscribed to the Google Groups
"flashcache-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
For more options, visit https://groups.google.com/groups/opt_out.
--
You received this message because you are subscribed to the Google Groups
"flashcache-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
For more options, visit https://groups.google.com/groups/opt_out.
--
You received this message because you are subscribed to the Google Groups "flashcache-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to flashcache-dev+***@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
Bill McGonigle
2013-08-13 16:20:16 UTC
Permalink
Post by Mohan Srinivasan
The IO scaling changes in master change the on-ssd cache format. But the
changes are compatible with the old format.
So, better performance/scaling with the new format? Is this a good
idea for new setups, or worth taking the time to reformat existing systems?

Thanks for your work!

-Bill
--
Bill McGonigle, Owner
BFC Computing, LLC
http://bfccomputing.com/
Telephone: +1.855.SW.LIBRE
Email, IM, VOIP: ***@bfccomputing.com
VCard: http://bfccomputing.com/vcard/bill.vcf
Social networks: bill_mcgonigle/bill.mcgonigle
--
You received this message because you are subscribed to the Google Groups "flashcache-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to flashcache-dev+***@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
Mohan Srinivasan
2013-08-13 16:36:22 UTC
Permalink
Hi

Sharib - No work done yet on cache resizing.

Bill - New setups should definitely use the latest changes. Old setups will benefit from the microlocks work (fine grained locking within flashcache), the 2 LRU Queue (better cache hit rates) and the various fixes. I did test to make sure that the new code works on old setups, but before you put it in production, might be a good idea to test on 1 host at your end too :)

mohan


________________________________
From: Bill McGonigle <***@bfccomputing.com>
To: flashcache-***@googlegroups.com
Cc: Mohan Srinivasan <***@yahoo.com>
Sent: Tuesday, August 13, 2013 9:20 AM
Subject: Re: [flashcache-dev] New flashcache branch and lots of facebook scaling changes committed...
Post by Mohan Srinivasan
The IO scaling changes in master change the on-ssd cache format. But the
changes are compatible with the old format.
So, better performance/scaling with the new format?  Is this a good
idea for new setups, or worth taking the time to reformat existing systems?

Thanks for your work!

-Bill
--
Bill McGonigle, Owner
BFC Computing, LLC
http://bfccomputing.com/
Telephone: +1.855.SW.LIBRE
Email, IM, VOIP: ***@bfccomputing.com
VCard: http://bfccomputing.com/vcard/bill.vcf
Social networks: bill_mcgonigle/bill.mcgonigle
--
You received this message because you are subscribed to the Google Groups "flashcache-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to flashcache-dev+***@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
--
You received this message because you are subscribed to the Google Groups "flashcache-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to flashcache-dev+***@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
Akos Gabriel
2013-08-14 09:24:38 UTC
Permalink
Great, thanks. How to migrate an old setup to the new format?

Thanks,
Akos
Post by Mohan Srinivasan
One more note on the changes just committed - The IO scaling changes in master change the on-ssd cache format. But the changes are compatible with the old format. The code detects when the existing format on-ssd is old and reverts to old layout seamlessly.
mohan
Sent: Tuesday, August 13, 2013 8:36 AM
Subject: [flashcache-dev] New flashcache branch and lots of facebook scaling changes committed...
Hi
1) I created a new stable_v2.1 branch.
2) In master, post stable_v2.1, I committed all of the changes from the facebook internal flashcache repository. These include changes for better IO scaling, fine grained (per cache set) locking of flashcache, improved replacement algorithm (2 Queue) and several fixes.
I will be updating the documentation detailing these changes over the next few days.
Please let me know if you see issues.
thanks
mohan
--
You received this message because you are subscribed to the Google Groups "flashcache-dev" group.
For more options, visit https://groups.google.com/groups/opt_out.
--
You received this message because you are subscribed to the Google Groups "flashcache-dev" group.
For more options, visit https://groups.google.com/groups/opt_out.
--
Regards,
Ákos Gábriel
--
You received this message because you are subscribed to the Google Groups "flashcache-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to flashcache-dev+***@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
Mohan Srinivasan
2013-08-14 14:49:42 UTC
Permalink
Hi Akos

The short answer is that to migrate to the new format would require a cache recreate. The format is significantly different that you cannot do a migration per se. Very briefly, there are 2 changes to the format - (1) selection of a cache set is completely randomized now, instead of the linear layout before (2) within a cache set, we have the option of chopping it up into smaller sub-sets.

I hope to document the new format (and all the other changes) within a few days. We also have a facebook engineering blog post in the works that should be published shortly that describes these, I am waiting on that so that I can link that into my documentation updates.

mohan


________________________________
From: Akos Gabriel <***@gmail.com>
To: flashcache-***@googlegroups.com
Sent: Wednesday, August 14, 2013 2:24 AM
Subject: Re: [flashcache-dev] New flashcache branch and lots of facebook scaling changes committed...



Great, thanks. How to migrate an old setup to the new format?

Thanks,
Akos


2013.08.13. dátummal, 17:48 időpontban Mohan Srinivasan írta:

One more note on the changes just committed - The IO scaling changes in master change the on-ssd cache format. But the changes are compatible with the old format. The code detects when the existing format on-ssd is old and reverts to old layout seamlessly.
Post by Mohan Srinivasan
mohan
________________________________
Sent: Tuesday, August 13, 2013 8:36 AM
Subject: [flashcache-dev] New flashcache branch and lots of facebook scaling changes committed...
Hi
1) I created a new stable_v2.1 branch.
2) In master, post stable_v2.1, I committed all of the changes from the facebook internal flashcache repository. These include changes for better IO scaling, fine grained (per cache set) locking of flashcache, improved replacement algorithm (2 Queue) and several fixes.
I will be updating the documentation detailing these changes over the next few days.
Please let me know if you see issues.
thanks
mohan
--
Post by Mohan Srinivasan
You received this message because you are subscribed to the Google Groups "flashcache-dev" group.
For more options, visit https://groups.google.com/groups/opt_out.
 
 
--
Post by Mohan Srinivasan
You received this message because you are subscribed to the Google Groups "flashcache-dev" group.
For more options, visit https://groups.google.com/groups/opt_out.
 
 
-- 
Regards,
Ákos Gábriel
--
You received this message because you are subscribed to the Google Groups "flashcache-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to flashcache-dev+***@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
--
You received this message because you are subscribed to the Google Groups "flashcache-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to flashcache-dev+***@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
Akos Gabriel
2013-08-14 15:22:19 UTC
Permalink
Hi Mohan,

Thanks for the answer. Can you please briefly describe how to recreate the cache? (or point me to the appropriate docs?)

Thanks,
Akos
Post by Mohan Srinivasan
Hi Akos
The short answer is that to migrate to the new format would require a cache recreate. The format is significantly different that you cannot do a migration per se. Very briefly, there are 2 changes to the format - (1) selection of a cache set is completely randomized now, instead of the linear layout before (2) within a cache set, we have the option of chopping it up into smaller sub-sets.
I hope to document the new format (and all the other changes) within a few days. We also have a facebook engineering blog post in the works that should be published shortly that describes these, I am waiting on that so that I can link that into my documentation updates.
mohan
Sent: Wednesday, August 14, 2013 2:24 AM
Subject: Re: [flashcache-dev] New flashcache branch and lots of facebook scaling changes committed...
Great, thanks. How to migrate an old setup to the new format?
Thanks,
Akos
Post by Mohan Srinivasan
One more note on the changes just committed - The IO scaling changes in master change the on-ssd cache format. But the changes are compatible with the old format. The code detects when the existing format on-ssd is old and reverts to old layout seamlessly.
mohan
Sent: Tuesday, August 13, 2013 8:36 AM
Subject: [flashcache-dev] New flashcache branch and lots of facebook scaling changes committed...
Hi
1) I created a new stable_v2.1 branch.
2) In master, post stable_v2.1, I committed all of the changes from the facebook internal flashcache repository. These include changes for better IO scaling, fine grained (per cache set) locking of flashcache, improved replacement algorithm (2 Queue) and several fixes.
I will be updating the documentation detailing these changes over the next few days.
Please let me know if you see issues.
thanks
mohan
--
You received this message because you are subscribed to the Google Groups "flashcache-dev" group.
For more options, visit https://groups.google.com/groups/opt_out.
--
You received this message because you are subscribed to the Google Groups "flashcache-dev" group.
For more options, visit https://groups.google.com/groups/opt_out.
--
Regards,
Ákos Gábriel
--
You received this message because you are subscribed to the Google Groups "flashcache-dev" group.
For more options, visit https://groups.google.com/groups/opt_out.
--
You received this message because you are subscribed to the Google Groups "flashcache-dev" group.
For more options, visit https://groups.google.com/groups/opt_out.
--
Regards,
Ákos Gábriel
--
You received this message because you are subscribed to the Google Groups "flashcache-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to flashcache-dev+***@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
Mohan Srinivasan
2013-08-14 15:34:46 UTC
Permalink
You have to do a dmsetup remove (if writeback, make sure everything is written out to disk), then do a flashcache_destroy followed by a flashcache_create. By default you will get the randomized cache set placement I mentioned.


________________________________
From: Akos Gabriel <***@gmail.com>
To: flashcache-***@googlegroups.com
Sent: Wednesday, August 14, 2013 8:22 AM
Subject: Re: [flashcache-dev] New flashcache branch and lots of facebook scaling changes committed...



Hi Mohan,

Thanks for the answer. Can you please briefly describe how to recreate the cache? (or point me to the appropriate docs?)

Thanks,
Akos

2013.08.14. dátummal, 16:49 időpontban Mohan Srinivasan írta:

Hi Akos
Post by Mohan Srinivasan
The short answer is that to migrate to the new format would require a cache recreate. The format is significantly different that you cannot do a migration per se. Very briefly, there are 2 changes to the format - (1) selection of a cache set is completely randomized now, instead of the linear layout before (2) within a cache set, we have the option of chopping it up into smaller sub-sets.
I hope to document the new format (and all the other changes) within a few days. We also have a facebook engineering blog post in the works that should be published shortly that describes these, I am waiting on that so that I can link that into my documentation updates.
mohan
________________________________
Sent: Wednesday, August 14, 2013 2:24 AM
Subject: Re: [flashcache-dev] New flashcache branch and lots of facebook scaling changes committed...
Great, thanks. How to migrate an old setup to the new format?
Thanks,
Akos
One more note on the changes just committed - The IO scaling changes in master change the on-ssd cache format. But the changes are compatible with the old format. The code detects when the existing format on-ssd is old and reverts to old layout seamlessly.
Post by Mohan Srinivasan
mohan
________________________________
Sent: Tuesday, August 13, 2013 8:36 AM
Subject: [flashcache-dev] New flashcache branch and lots of facebook scaling changes committed...
Hi
1) I created a new stable_v2.1 branch.
2) In master, post stable_v2.1, I committed all of the changes from the facebook internal flashcache repository. These include changes for better IO scaling, fine grained (per cache set) locking of flashcache, improved replacement algorithm (2 Queue) and several fixes.
I will be updating the documentation detailing these changes over the next few days.
Please let me know if you see issues.
thanks
mohan
--
Post by Mohan Srinivasan
Post by Mohan Srinivasan
You received this message because you are subscribed to the Google Groups "flashcache-dev" group.
For more options, visit https://groups.google.com/groups/opt_out.
 
 
--
Post by Mohan Srinivasan
Post by Mohan Srinivasan
You received this message because you are subscribed to the Google Groups "flashcache-dev" group.
For more options, visit https://groups.google.com/groups/opt_out.
 
 
-- 
Regards,
Ákos Gábriel
--
Post by Mohan Srinivasan
You received this message because you are subscribed to the Google Groups "flashcache-dev" group.
For more options, visit https://groups.google.com/groups/opt_out.
--
Post by Mohan Srinivasan
You received this message because you are subscribed to the Google Groups "flashcache-dev" group.
For more options, visit https://groups.google.com/groups/opt_out.
 
 
-- 
Regards,
Ákos Gábriel
--
You received this message because you are subscribed to the Google Groups "flashcache-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to flashcache-dev+***@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
--
You received this message because you are subscribed to the Google Groups "flashcache-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to flashcache-dev+***@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
Akos Gabriel
2013-08-14 15:38:19 UTC
Permalink
make sure everything is written to disk : means I must not interrupt the execution, need to wait till it returns, right?

thanks,
Akos
Post by Mohan Srinivasan
You have to do a dmsetup remove (if writeback, make sure everything is written out to disk), then do a flashcache_destroy followed by a flashcache_create. By default you will get the randomized cache set placement I mentioned.
Sent: Wednesday, August 14, 2013 8:22 AM
Subject: Re: [flashcache-dev] New flashcache branch and lots of facebook scaling changes committed...
Hi Mohan,
Thanks for the answer. Can you please briefly describe how to recreate the cache? (or point me to the appropriate docs?)
Thanks,
Akos
Post by Mohan Srinivasan
Hi Akos
The short answer is that to migrate to the new format would require a cache recreate. The format is significantly different that you cannot do a migration per se. Very briefly, there are 2 changes to the format - (1) selection of a cache set is completely randomized now, instead of the linear layout before (2) within a cache set, we have the option of chopping it up into smaller sub-sets.
I hope to document the new format (and all the other changes) within a few days. We also have a facebook engineering blog post in the works that should be published shortly that describes these, I am waiting on that so that I can link that into my documentation updates.
mohan
Sent: Wednesday, August 14, 2013 2:24 AM
Subject: Re: [flashcache-dev] New flashcache branch and lots of facebook scaling changes committed...
Great, thanks. How to migrate an old setup to the new format?
Thanks,
Akos
Post by Mohan Srinivasan
One more note on the changes just committed - The IO scaling changes in master change the on-ssd cache format. But the changes are compatible with the old format. The code detects when the existing format on-ssd is old and reverts to old layout seamlessly.
mohan
Sent: Tuesday, August 13, 2013 8:36 AM
Subject: [flashcache-dev] New flashcache branch and lots of facebook scaling changes committed...
Hi
1) I created a new stable_v2.1 branch.
2) In master, post stable_v2.1, I committed all of the changes from the facebook internal flashcache repository. These include changes for better IO scaling, fine grained (per cache set) locking of flashcache, improved replacement algorithm (2 Queue) and several fixes.
I will be updating the documentation detailing these changes over the next few days.
Please let me know if you see issues.
thanks
mohan
--
You received this message because you are subscribed to the Google Groups "flashcache-dev" group.
For more options, visit https://groups.google.com/groups/opt_out.
--
You received this message because you are subscribed to the Google Groups "flashcache-dev" group.
For more options, visit https://groups.google.com/groups/opt_out.
--
Regards,
Ákos Gábriel
--
You received this message because you are subscribed to the Google Groups "flashcache-dev" group.
For more options, visit https://groups.google.com/groups/opt_out.
--
You received this message because you are subscribed to the Google Groups "flashcache-dev" group.
For more options, visit https://groups.google.com/groups/opt_out.
--
Regards,
Ákos Gábriel
--
You received this message because you are subscribed to the Google Groups "flashcache-dev" group.
For more options, visit https://groups.google.com/groups/opt_out.
--
You received this message because you are subscribed to the Google Groups "flashcache-dev" group.
For more options, visit https://groups.google.com/groups/opt_out.
--
Regards,
Ákos Gábriel
--
You received this message because you are subscribed to the Google Groups "flashcache-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to flashcache-dev+***@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
Mohan Srinivasan
2013-08-14 15:53:53 UTC
Permalink
If writeback, this is what you need.

1) Turn off the fast_remove sysctl.
2) unmount and do a dmsetup remove. dmsetup remove will not return until all dirty blocks are written to disk.
3) flashcache_destroy, flashcache_create with the new module.


________________________________
From: Akos Gabriel <***@gmail.com>
To: flashcache-***@googlegroups.com
Sent: Wednesday, August 14, 2013 8:38 AM
Subject: Re: [flashcache-dev] New flashcache branch and lots of facebook scaling changes committed...



make sure everything is written to disk : means I must not interrupt the execution, need to wait till it returns, right?

thanks,
Akos


2013.08.14. dátummal, 17:34 idõpontban Mohan Srinivasan írta:

You have to do a dmsetup remove (if writeback, make sure everything is written out to disk), then do a flashcache_destroy followed by a flashcache_create. By default you will get the randomized cache set placement I mentioned.
Post by Mohan Srinivasan
________________________________
Sent: Wednesday, August 14, 2013 8:22 AM
Subject: Re: [flashcache-dev] New flashcache branch and lots of facebook scaling changes committed...
Hi Mohan,
Thanks for the answer. Can you please briefly describe how to recreate the cache? (or point me to the appropriate docs?)
Thanks,
Akos
Hi Akos
Post by Mohan Srinivasan
The short answer is that to migrate to the new format would require a cache recreate. The format is significantly different that you cannot do a migration per se. Very briefly, there are 2 changes to the format - (1) selection of a cache set is completely randomized now, instead of the linear layout before (2) within a cache set, we have the option of chopping it up into smaller sub-sets.
I hope to document the new format (and all the other changes) within a few days. We also have a facebook engineering blog post in the works that should be published shortly that describes these, I am waiting on that so that I can link that into my documentation updates.
mohan
________________________________
Sent: Wednesday, August 14, 2013 2:24 AM
Subject: Re: [flashcache-dev] New flashcache branch and lots of facebook scaling changes committed...
Great, thanks. How to migrate an old setup to the new format?
Thanks,
Akos
One more note on the changes just committed - The IO scaling changes in master change the on-ssd cache format. But the changes are compatible with the old format. The code detects when the existing format on-ssd is old and reverts to old layout seamlessly.
Post by Mohan Srinivasan
mohan
________________________________
Sent: Tuesday, August 13, 2013 8:36 AM
Subject: [flashcache-dev] New flashcache branch and lots of facebook scaling changes committed...
Hi
1) I created a new stable_v2.1 branch.
2) In master, post stable_v2.1, I committed all of the changes from the facebook internal flashcache repository. These include changes for better IO scaling, fine grained (per cache set) locking of flashcache, improved replacement algorithm (2 Queue) and several fixes.
I will be updating the documentation detailing these changes over the next few days.
Please let me know if you see issues.
thanks
mohan
--
Post by Mohan Srinivasan
Post by Mohan Srinivasan
Post by Mohan Srinivasan
You received this message because you are subscribed to the Google Groups "flashcache-dev" group.
For more options, visit https://groups.google.com/groups/opt_out.
 
 
--
Post by Mohan Srinivasan
Post by Mohan Srinivasan
Post by Mohan Srinivasan
You received this message because you are subscribed to the Google Groups "flashcache-dev" group.
For more options, visit https://groups.google.com/groups/opt_out.
 
 
-- 
Regards,
Ákos Gábriel
--
Post by Mohan Srinivasan
Post by Mohan Srinivasan
You received this message because you are subscribed to the Google Groups "flashcache-dev" group.
For more options, visit https://groups.google.com/groups/opt_out.
--
Post by Mohan Srinivasan
Post by Mohan Srinivasan
You received this message because you are subscribed to the Google Groups "flashcache-dev" group.
For more options, visit https://groups.google.com/groups/opt_out.
 
 
-- 
Regards,
Ákos Gábriel
--
Post by Mohan Srinivasan
You received this message because you are subscribed to the Google Groups "flashcache-dev" group.
For more options, visit https://groups.google.com/groups/opt_out.
 
 
--
Post by Mohan Srinivasan
You received this message because you are subscribed to the Google Groups "flashcache-dev" group.
For more options, visit https://groups.google.com/groups/opt_out.
 
 
-- 
Regards,
Ákos Gábriel
--
You received this message because you are subscribed to the Google Groups "flashcache-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to flashcache-dev+***@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
--
You received this message because you are subscribed to the Google Groups "flashcache-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to flashcache-dev+***@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
hs
2013-08-20 10:04:36 UTC
Permalink
*Hi mohan,*
*
*
*Thanks for your great work.*
*
*
*And I have a question as follows:*
*
*
*I studied the source code about flashcache 2.0,and I want to add some
replacement algorithm into it.Such as:LFU,LIRS.*
*
*
*Because LFU is similar to LRU,so I inplemented LFU without much
diffculties,,and did some test to evaluate its performance.*
*
*
*But I met a problem about LIRS algorithm,as the huge difference between
LIRS and LRU.*
*
*
*I wonder whether it is possible to add LIRS algorithm into flashcache.*
*
*
*If it is possible,could you give me some advice.*
*
*
*I am looking forward your reply.*
*
*
*Thank you*
*
*
*hs*
Post by Mohan Srinivasan
Hi
1) I created a new stable_v2.1 branch.
2) In master, post stable_v2.1, I committed all of the changes from the
facebook internal flashcache repository. These include changes for better
IO scaling, fine grained (per cache set) locking of flashcache, improved
replacement algorithm (2 Queue) and several fixes.
I will be updating the documentation detailing these changes over the next few days.
Please let me know if you see issues.
thanks
mohan
--
You received this message because you are subscribed to the Google Groups "flashcache-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to flashcache-dev+***@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
Loading...