110 lines
21 KiB
HTML
110 lines
21 KiB
HTML
<html><head><meta http-equiv="X-UA-Compatible" content="IE=edge" /><link rel="shortcut icon" href="../icons/favicon.ico" /><style type="text/css">.OH_CodeSnippetContainerTabLeftActive, .OH_CodeSnippetContainerTabLeft,.OH_CodeSnippetContainerTabLeftDisabled { }.OH_CodeSnippetContainerTabRightActive, .OH_CodeSnippetContainerTabRight,.OH_CodeSnippetContainerTabRightDisabled { }.OH_footer { }</style><link rel="stylesheet" type="text/css" href="../styles/branding.css" /><link rel="stylesheet" type="text/css" href="../styles/branding-en-US.css" /><script type="text/javascript" src="../scripts/branding.js"> </script><meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /><title>Microsoft.Research.DryadLinq Namespace</title><meta name="Language" content="en-us" /><meta name="System.Keywords" content="Microsoft.Research.DryadLinq namespace" /><meta name="Microsoft.Help.F1" content="Microsoft.Research.DryadLinq" /><meta name="Microsoft.Help.Id" content="N:Microsoft.Research.DryadLinq" /><meta name="Description" content="The root namespace for DryadLinq client programs" /><meta name="Microsoft.Help.ContentType" content="Reference" /><meta name="BrandingAware" content="true" /><meta name="container" content="Microsoft.Research.DryadLinq" /><meta name="file" content="efe6507e-9fd8-bbd3-8227-fd6ba9e289c1" /><meta name="guid" content="efe6507e-9fd8-bbd3-8227-fd6ba9e289c1" /></head><body onload="OnLoad('cs')"><input type="hidden" id="userDataCache" class="userDataStyle" /><div class="OH_outerDiv"><div class="OH_outerContent"><table class="TitleTable"><tr><td class="OH_tdTitleColumn">Microsoft.Research.DryadLinq Namespace</td><td class="OH_tdRunningTitleColumn">DryadLINQ documentation</td></tr></table><div id="mainSection"><div id="mainBody"><span class="introStyle"></span><div class="summary">
|
||
The root namespace for DryadLinq client programs
|
||
</div><div class="OH_CollapsibleAreaRegion"><div class="OH_regiontitle">Classes</div><div class="OH_CollapsibleArea_HrDiv"><hr class="OH_CollapsibleArea_Hr" /></div></div><div class="OH_clear"> </div><table id="typeList" class="members"><tr><th class="ps_iconColumn">
|
||
|
||
</th><th>Class</th><th>Description</th></tr><tr data="class; public"><td><img src="../icons/pubclass.gif" alt="Public class" title="Public class" /></td><td><a href="d3237c01-5085-45a9-fa72-8e489a9bf480.htm">AssociativeAttribute</a></td><td><div class="summary">
|
||
Indicates that a method is an associative aggregation method. The argument to the
|
||
constructor must be of type IAssociative. The computation of the method annotated
|
||
by this attribute can be decomposed to a sequence of calls to the Seed and
|
||
RecursiveAccumulate methods.
|
||
</div></td></tr><tr data="class; public"><td><img src="../icons/pubclass.gif" alt="Public class" title="Public class" /></td><td><a href="1842f60d-35dc-a934-cc36-dfa0cba934e1.htm">CustomDryadLinqSerializerAttribute</a></td><td><div class="summary">
|
||
Provides a user-defined serialization method for a .NET type.
|
||
</div></td></tr><tr data="class; public"><td><img src="../icons/pubclass.gif" alt="Public class" title="Public class" /></td><td><a href="e30d9823-523d-4504-b321-4c990c768d6d.htm">DataProvider</a></td><td><div class="summary">
|
||
DataProvider provides an abstraction for different data backends. New data storage backends
|
||
could be added by subclassing this class.
|
||
</div></td></tr><tr data="class; public"><td><img src="../icons/pubclass.gif" alt="Public class" title="Public class" /></td><td><a href="49fcc552-44b3-1cd7-9e8d-0182923c9af4.htm">DecomposableAttribute</a></td><td><div class="summary">
|
||
Indicates that a method can be decomposed to multiple methods. The argument to the
|
||
constructor must be of type IDecomposable. The computation of the method annotated
|
||
by this attribute can be decomposed to a sequence of calls to the Seed, Accumulate,
|
||
RecursiveAccumulate methods and a FinalReduce.
|
||
</div></td></tr><tr data="class; public"><td><img src="../icons/pubclass.gif" alt="Public class" title="Public class" /></td><td><a href="e2f6b120-72cd-d97c-d0c9-9ab1b4abb5b5.htm">DryadLinqBinaryReader</a></td><td><div class="summary">
|
||
DryadLinqBinaryReader is the main interface for user provided custom serializers.
|
||
It is also used for DryadLINQ internal autoserialization to read primitive types.
|
||
</div></td></tr><tr data="class; public"><td><img src="../icons/pubclass.gif" alt="Public class" title="Public class" /></td><td><a href="46c3c8f6-eac2-96d5-3ff0-d837669e9557.htm">DryadLinqBinaryWriter</a></td><td><div class="summary">
|
||
DryadLinqBinaryWriter is the main interface for user provided custom serializers.
|
||
It is also used for DryadLINQ internal autoserialization to write primitive types.
|
||
</div></td></tr><tr data="class; public"><td><img src="../icons/pubclass.gif" alt="Public class" title="Public class" /></td><td><a href="093a0025-a8c4-d1e2-a289-d3863b1a845f.htm">DryadLinqContext</a></td><td><div class="summary">
|
||
Represents the context necessary to prepare and execute a DryadLinq Query,
|
||
</div></td></tr><tr data="class; public"><td><img src="../icons/pubclass.gif" alt="Public class" title="Public class" /></td><td><a href="692e14c6-46ed-baaf-6b0f-b36cb74fe616.htm">DryadLinqException</a></td><td><div class="summary">
|
||
The exception that is thrown by DryadLINQ.
|
||
</div></td></tr><tr data="class; public"><td><img src="../icons/pubclass.gif" alt="Public class" title="Public class" /></td><td><a href="d83d6168-067a-d431-aaf5-3d9515eff360.htm">DryadLinqExtension</a></td><td><div class="summary">
|
||
Defines some useful operators that are commonly used in applications. The
|
||
operators are defined using the basic DryadLINQ operators. This class
|
||
also shows how a user library can be defined.
|
||
</div></td></tr><tr data="class; public"><td><img src="../icons/pubclass.gif" alt="Public class" title="Public class" /></td><td><a href="1286706e-b13a-d4da-31b6-a8e9095728c7.htm">DryadLinqJobInfo</a></td><td><div class="summary">
|
||
Represents the current state of a set of DryadLINQ jobs that have already been
|
||
submitted for execution. A DryadLinqJobInfo object is returned after a job is
|
||
submitted for execution.
|
||
</div></td></tr><tr data="class; public"><td><img src="../icons/pubclass.gif" alt="Public class" title="Public class" /></td><td><a href="85ce5a97-d2bb-bd50-ca58-f017d3769b7c.htm">DryadLinqLog</a></td><td><div class="summary">
|
||
DryadLINQ logging API.
|
||
</div></td></tr><tr data="class; public"><td><img src="../icons/pubclass.gif" alt="Public class" title="Public class" /></td><td><a href="5e3ae825-91af-0ff7-6ae4-f98f18bd6e58.htm">DryadLinqMetaData</a></td><td><div class="summary">
|
||
The metadata associated with a DryadLINQ dataset.
|
||
</div></td></tr><tr data="class; public"><td><img src="../icons/pubclass.gif" alt="Public class" title="Public class" /></td><td><a href="6d67ec6b-96ed-0a8b-0ad4-1c0568c7bf91.htm">DryadLinqQueryable</a></td><td><div class="summary">
|
||
This class extends LINQ with a set of new operators that are specific to DryadLINQ.
|
||
The new operators includes partitioning operators (HashPartition and RangePartition)
|
||
and the Apply operator that enables stateful transformations on datasets.
|
||
</div></td></tr><tr data="class; public"><td><img src="../icons/pubclass.gif" alt="Public class" title="Public class" /></td><td><a href="562bb8ee-21e0-0f88-c57c-64a244256648.htm">DryadLinqStreamInfo</a></td><td><div class="summary">
|
||
Basic information of a dataset.
|
||
</div></td></tr><tr data="class; public"><td><img src="../icons/pubclass.gif" alt="Public class" title="Public class" /></td><td><a href="13468856-16ae-cd23-9019-d940bd07e7a6.htm">DryadLinqYarnCluster</a></td><td><div class="summary">
|
||
The interface for a YARN native cluster.
|
||
</div></td></tr><tr data="class; public"><td><img src="../icons/pubclass.gif" alt="Public class" title="Public class" /></td><td><a href="a49d01fb-8518-8fe8-02b4-7fd06cec7ed4.htm">GenericAssociative<span id="LST48CE9A7A_0"></span><script type="text/javascript">AddLanguageSpecificTextSet("LST48CE9A7A_0?cs=<|vb=(Of |cpp=<|nu=(|fs=<'");</script>TAssoc, TAccumulate<span id="LST48CE9A7A_1"></span><script type="text/javascript">AddLanguageSpecificTextSet("LST48CE9A7A_1?cs=>|vb=)|cpp=>|nu=)|fs=>");</script></a></td><td><div class="summary">
|
||
A helper class for calling IAssociative methods more efficiently. It is used in
|
||
auto-generated vertex code. A DryadLINQ user should not need to use this class directly.
|
||
</div></td></tr><tr data="class; public"><td><img src="../icons/pubclass.gif" alt="Public class" title="Public class" /></td><td><a href="11c77de4-bb21-4879-6a37-daaf58adb3cf.htm">GenericDecomposable<span id="LST48CE9A7A_2"></span><script type="text/javascript">AddLanguageSpecificTextSet("LST48CE9A7A_2?cs=<|vb=(Of |cpp=<|nu=(|fs=<'");</script>TDecomposable, TSource, TAccumulate, TResult<span id="LST48CE9A7A_3"></span><script type="text/javascript">AddLanguageSpecificTextSet("LST48CE9A7A_3?cs=>|vb=)|cpp=>|nu=)|fs=>");</script></a></td><td><div class="summary">
|
||
A helper class for calling IDecomposable methods more efficiently. It is used in
|
||
auto-generated vertex code. A DryadLINQ user should not need to use this class directly.
|
||
</div></td></tr><tr data="class; public"><td><img src="../icons/pubclass.gif" alt="Public class" title="Public class" /></td><td><a href="d38002f5-8220-c873-ca1b-5c99e45e5007.htm">NullableAttribute</a></td><td><div class="summary">
|
||
The Nullable attribute specifies if a field is nullable. The information is used by DryadLINQ
|
||
serialization. DryadLINQ serialization by default treats all fields not nullable.
|
||
</div></td></tr><tr data="class; public"><td><img src="../icons/pubclass.gif" alt="Public class" title="Public class" /></td><td><a href="f48da9e0-e74d-da3a-ca41-d3afe376271d.htm">ResourceAttribute</a></td><td><div class="summary">
|
||
The Resource attribute is used to specify the computation cost of a user defined
|
||
function (UDF). IsStateful asserts that the function is stateful; IsExpensive
|
||
asserts that the function is expensive to compute. The information is useful in
|
||
generating better execution plan. For example, expensive associative aggregation
|
||
functions enables the use of multiple aggregation layers.
|
||
</div></td></tr></table><div class="OH_CollapsibleAreaRegion"><div class="OH_regiontitle">Structures</div><div class="OH_CollapsibleArea_HrDiv"><hr class="OH_CollapsibleArea_Hr" /></div></div><div class="OH_clear"> </div><table id="typeList" class="members"><tr><th class="ps_iconColumn">
|
||
|
||
</th><th>Structure</th><th>Description</th></tr><tr data="structure; public"><td><img src="../icons/pubstructure.gif" alt="Public structure" title="Public structure" /></td><td><a href="28af6b37-22b3-448f-cf9f-ea15b5aa90d6.htm">ForkTuple<span id="LST48CE9A7A_4"></span><script type="text/javascript">AddLanguageSpecificTextSet("LST48CE9A7A_4?cs=<|vb=(Of |cpp=<|nu=(|fs=<'");</script>T1, T2<span id="LST48CE9A7A_5"></span><script type="text/javascript">AddLanguageSpecificTextSet("LST48CE9A7A_5?cs=>|vb=)|cpp=>|nu=)|fs=>");</script></a></td><td><div class="summary">
|
||
Represents a pair of elements that may not have valid values.
|
||
</div></td></tr><tr data="structure; public"><td><img src="../icons/pubstructure.gif" alt="Public structure" title="Public structure" /></td><td><a href="07206258-2e2e-6aa3-4c79-0c0674875849.htm">ForkTuple<span id="LST48CE9A7A_6"></span><script type="text/javascript">AddLanguageSpecificTextSet("LST48CE9A7A_6?cs=<|vb=(Of |cpp=<|nu=(|fs=<'");</script>T1, T2, T3<span id="LST48CE9A7A_7"></span><script type="text/javascript">AddLanguageSpecificTextSet("LST48CE9A7A_7?cs=>|vb=)|cpp=>|nu=)|fs=>");</script></a></td><td><div class="summary">
|
||
Represents a tuple of three elements that may not have valid values.
|
||
</div></td></tr><tr data="structure; public"><td><img src="../icons/pubstructure.gif" alt="Public structure" title="Public structure" /></td><td><a href="3d9c6034-3028-09ed-9ec6-6af9101c0f52.htm">ForkValue<span id="LST48CE9A7A_8"></span><script type="text/javascript">AddLanguageSpecificTextSet("LST48CE9A7A_8?cs=<|vb=(Of |cpp=<|nu=(|fs=<'");</script>T<span id="LST48CE9A7A_9"></span><script type="text/javascript">AddLanguageSpecificTextSet("LST48CE9A7A_9?cs=>|vb=)|cpp=>|nu=)|fs=>");</script></a></td><td><div class="summary">
|
||
Represents an element that may not have valid value.
|
||
</div></td></tr><tr data="structure; public"><td><img src="../icons/pubstructure.gif" alt="Public structure" title="Public structure" /></td><td><a href="b077f955-9bc6-1e0b-e710-00cc784bccf1.htm">LineRecord</a></td><td><div class="summary">
|
||
The DryadLINQ type to represent a line of text.
|
||
</div></td></tr><tr data="structure; public"><td><img src="../icons/pubstructure.gif" alt="Public structure" title="Public structure" /></td><td><a href="4937c0d0-14bc-0943-1166-e434ec5c8dfd.htm">Pair<span id="LST48CE9A7A_10"></span><script type="text/javascript">AddLanguageSpecificTextSet("LST48CE9A7A_10?cs=<|vb=(Of |cpp=<|nu=(|fs=<'");</script>T1, T2<span id="LST48CE9A7A_11"></span><script type="text/javascript">AddLanguageSpecificTextSet("LST48CE9A7A_11?cs=>|vb=)|cpp=>|nu=)|fs=>");</script></a></td><td><div class="summary">
|
||
Represents a key/value pair. Very similar to KeyValuePair, but adds a few more methods.
|
||
</div></td></tr></table><div class="OH_CollapsibleAreaRegion"><div class="OH_regiontitle">Interfaces</div><div class="OH_CollapsibleArea_HrDiv"><hr class="OH_CollapsibleArea_Hr" /></div></div><div class="OH_clear"> </div><table id="typeList" class="members"><tr><th class="ps_iconColumn">
|
||
|
||
</th><th>Interface</th><th>Description</th></tr><tr data="interface; public"><td><img src="../icons/pubinterface.gif" alt="Public interface" title="Public interface" /></td><td><a href="65be1ae5-77ef-6de4-cd1f-586d32aeb383.htm">DryadLinqCluster</a></td><td><div class="summary">
|
||
Base interface for cluster types that the DryadLinqContext constructor can accept.
|
||
</div></td></tr><tr data="interface; public"><td><img src="../icons/pubinterface.gif" alt="Public interface" title="Public interface" /></td><td><a href="8ac439be-5f1e-cce5-eea2-11de7aa46929.htm">IAssociative<span id="LST48CE9A7A_12"></span><script type="text/javascript">AddLanguageSpecificTextSet("LST48CE9A7A_12?cs=<|vb=(Of |cpp=<|nu=(|fs=<'");</script>TAccumulate<span id="LST48CE9A7A_13"></span><script type="text/javascript">AddLanguageSpecificTextSet("LST48CE9A7A_13?cs=>|vb=)|cpp=>|nu=)|fs=>");</script></a></td><td><div class="summary">
|
||
Defines the DryadLINQ interface for associative accumulator.
|
||
</div></td></tr><tr data="interface; public"><td><img src="../icons/pubinterface.gif" alt="Public interface" title="Public interface" /></td><td><a href="28f10369-ddd1-2090-d4e2-16e5c7b34795.htm">IDecomposable<span id="LST48CE9A7A_14"></span><script type="text/javascript">AddLanguageSpecificTextSet("LST48CE9A7A_14?cs=<|vb=(Of |cpp=<|nu=(|fs=<'");</script>TSource, TAccumulate, TResult<span id="LST48CE9A7A_15"></span><script type="text/javascript">AddLanguageSpecificTextSet("LST48CE9A7A_15?cs=>|vb=)|cpp=>|nu=)|fs=>");</script></a></td><td><div class="summary">
|
||
Defines the DryadLINQ interface for decomposable functions. It allows a function to be
|
||
decomposed into the composition of several functions that can be executed more efficiently.
|
||
</div></td></tr><tr data="interface; public"><td><img src="../icons/pubinterface.gif" alt="Public interface" title="Public interface" /></td><td><a href="21f7cc37-f1d3-ac9c-873a-e5e80c2b6f3c.htm">IDryadLinqSerializer<span id="LST48CE9A7A_16"></span><script type="text/javascript">AddLanguageSpecificTextSet("LST48CE9A7A_16?cs=<|vb=(Of |cpp=<|nu=(|fs=<'");</script>T<span id="LST48CE9A7A_17"></span><script type="text/javascript">AddLanguageSpecificTextSet("LST48CE9A7A_17?cs=>|vb=)|cpp=>|nu=)|fs=>");</script></a></td><td><div class="summary">
|
||
The interface for providing user-defined serialization for a .NET type.
|
||
If a class T implements DryadLinqSerializer{T}, DryadLinq will use the
|
||
read and write methods of the class to do serialization.
|
||
</div></td></tr><tr data="interface; public"><td><img src="../icons/pubinterface.gif" alt="Public interface" title="Public interface" /></td><td><a href="721bf2e3-dc55-bf0a-0873-d5a6b711d57a.htm">IKeyedMultiQueryable<span id="LST48CE9A7A_18"></span><script type="text/javascript">AddLanguageSpecificTextSet("LST48CE9A7A_18?cs=<|vb=(Of |cpp=<|nu=(|fs=<'");</script>T, K<span id="LST48CE9A7A_19"></span><script type="text/javascript">AddLanguageSpecificTextSet("LST48CE9A7A_19?cs=>|vb=)|cpp=>|nu=)|fs=>");</script></a></td><td><div class="summary">
|
||
The interface to access a collection of IQueryable{T} instances. Each IQueryable{T}
|
||
contains only elements of the same key. The IQueryable{T}s are indexed by a set of keys.
|
||
</div></td></tr><tr data="interface; public"><td><img src="../icons/pubinterface.gif" alt="Public interface" title="Public interface" /></td><td><a href="1a74f3d6-9e33-6ea7-b3c2-2bf6ecbc5d89.htm">IMultiQueryable</a></td><td><div class="summary">
|
||
The base interface to access a collection of IQueryable instances. The
|
||
DryadLINQ Fork operator returns a value that implements this interface.
|
||
</div></td></tr><tr data="interface; public"><td><img src="../icons/pubinterface.gif" alt="Public interface" title="Public interface" /></td><td><a href="2f4da221-3f06-8d95-c5ef-20eef2574124.htm">IMultiQueryable<span id="LST48CE9A7A_20"></span><script type="text/javascript">AddLanguageSpecificTextSet("LST48CE9A7A_20?cs=<|vb=(Of |cpp=<|nu=(|fs=<'");</script>T1, T2<span id="LST48CE9A7A_21"></span><script type="text/javascript">AddLanguageSpecificTextSet("LST48CE9A7A_21?cs=>|vb=)|cpp=>|nu=)|fs=>");</script></a></td><td><div class="summary">
|
||
The interface to access a collection of two IQueryable{T} instances.
|
||
</div></td></tr><tr data="interface; public"><td><img src="../icons/pubinterface.gif" alt="Public interface" title="Public interface" /></td><td><a href="cfec878f-f774-d543-961d-ea6185530786.htm">IMultiQueryable<span id="LST48CE9A7A_22"></span><script type="text/javascript">AddLanguageSpecificTextSet("LST48CE9A7A_22?cs=<|vb=(Of |cpp=<|nu=(|fs=<'");</script>T1, T2, T3<span id="LST48CE9A7A_23"></span><script type="text/javascript">AddLanguageSpecificTextSet("LST48CE9A7A_23?cs=>|vb=)|cpp=>|nu=)|fs=>");</script></a></td><td><div class="summary">
|
||
The interface to access a collection of three IQueryable{T} instances.
|
||
</div></td></tr></table><div class="OH_CollapsibleAreaRegion"><div class="OH_regiontitle">Enumerations</div><div class="OH_CollapsibleArea_HrDiv"><hr class="OH_CollapsibleArea_Hr" /></div></div><div class="OH_clear"> </div><table id="typeList" class="members"><tr><th class="ps_iconColumn">
|
||
|
||
</th><th>Enumeration</th><th>Description</th></tr><tr data="enumeration; public"><td><img src="../icons/pubenumeration.gif" alt="Public enumeration" title="Public enumeration" /></td><td><a href="9266635e-c3ea-7e66-528b-f44b55a2daca.htm">CompressionScheme</a></td><td><div class="summary">
|
||
Specifies whether to compress the output of a DryadLINQ vertex.
|
||
</div></td></tr><tr data="enumeration; public"><td><img src="../icons/pubenumeration.gif" alt="Public enumeration" title="Public enumeration" /></td><td><a href="e652bfe9-c973-2744-87c1-e63cf9db4f56.htm">ExecutorKind</a></td><td><div class="summary">
|
||
The executor to run DryadLINQ jobs. The current release only supports Dryad.
|
||
</div></td></tr><tr data="enumeration; public"><td><img src="../icons/pubenumeration.gif" alt="Public enumeration" title="Public enumeration" /></td><td><a href="db48824c-598d-a2a7-24db-a422d19467b1.htm">PlatformKind</a></td><td><div class="summary">
|
||
The service platforms where you can run DryadLINQ.
|
||
</div></td></tr><tr data="enumeration; public"><td><img src="../icons/pubenumeration.gif" alt="Public enumeration" title="Public enumeration" /></td><td><a href="0bebfe2a-dcec-f4bd-c2e6-0754204f4192.htm">QueryLoggingLevel</a></td><td><div class="summary">
|
||
Logging levels for DryadLinqQuery runtime.
|
||
</div></td></tr></table></div></div></div></div><div id="OH_footer" class="OH_footer"> </div></body></html> |